API Docs · Repo · NPM · Plugins · InterroBot
InterroBot plugins transform your web crawler into, well... anything you want. With unrestricted API crawl data access, Interrobot plugins can bring your website analysis concepts to life.
InterroBot plugins are simple HTML/JS/CSS pages that turn crawl data into insights, stunning visualizations, and interactive dashboards. With our flexible API, you can create custom plugins that analyze website content across entire domains, connecting with analytics, LLMs, or your favorite SaaS for deeper insights.
The InterroBot plugin ecosystem is designed for power users. Whether you're building proprietary tools, developing plugins for clients, or contributing to the open-source community, InterroBot plugins adapt to your needs. Available for Windows 10/11, macOS, Linux, and Android, InterroBot plugins are everywhere you need them.
InterroBot hosts an iframe of your webpage and exposes an API from which you can pull data down for analysis.
If you're familiar with vanilla TypeScript or JavaScript, creating a custom plugin script for InterroBot is remarkably straight forward. First you start with a bare-bones HTML file and a script extending the Plugin base class.
// TypeScript vs. JavaScript, both are fine. See examples.
import { Plugin } from "./src/ts/core/plugin";
class BasicExamplePlugin extends Plugin {
static meta = {
"title": "Example Plugin",
"category": "Example",
"version": "1.0.0",
"author": "InterroBot",
"synopsis": `a basic plugin example`,
"description": `This example is as simple as it gets.`,
};
constructor() {
super();
// index() has nothing to do with the crawl index, btw. it is
// the plugin index (think index.html), a view that shows by
// default, and would generally consist of a form or visualization.
this.index();
}
}
// configure to initialize when the page is ready
Plugin.initialize(BasicExamplePlugin);BasicExamplePlugin will not do much at this point, but it will load and run the default index() behavior.
You can, of course, override the default index() behavior, rendering your page however you wish.
protected async index() {
// add your form and supporting HTML
this.render(`<div>HTML</div>`);
// initialize the plugin within InterroBot, from within iframe
await this.initData({}, []);
// add handlers to the form
const button = document.querySelector("button");
button.addEventListener("click", async (ev) => {
await this.process(); // where process() is a form handler
});
}The process() method called above would be where you process data. Query the crawl index with
this.search(), and stream the results — pagination is handled for you, and you can break out
of the loop at any point to stop fetching.
protected async process() {
// queries are exactly as you'd type them into InterroBot search.
// id and url come with the base model, everything else costs time.
// here, I just grab the "name" field (the page title)
const titleWords = new Map();
for await (const result of this.search("headers: text/html",
{ fields: ["name"], includeExternal: false })) {
const terms = result.name.trim().split(/[\s\-—]+/g);
terms.forEach(term => titleWords.set(term,
(titleWords.get(term) ?? 0) + 1));
}
// call for HTML presentation of titleWords with processing complete
await this.report(titleWords);
}Behind this.search() sits the lower-level API: build an InterroBot.Core.SearchQuery for any
project, and either stream it (InterroBot.Core.Search.results(query)) or run a result handler
callback over it with progress events and result caching (InterroBot.Core.Search.execute(query, resultsMap, resultHandler, options)). this.search() is the recommended path; reach for execute
when you want cached replay of the result set across reruns of a report.
For working plugins and more ideas getting started, check out the examples directory.
InterroBot's API provides developers with access to crawled data, enabling analysis and useful customizations. This data allows you to create insightful visualizations, perform complex analysis, or build interactive tools. Whether you're tracking SEO metrics, analyzing content structures, or developing custom reporting tools, our API offers the flexibility and depth you need. Below is an overview of the key data points available, organized by API endpoint:
Retrieves a list of projects using the Plugin API.
Optional Fields
| Field | Description |
|---|---|
| created | ISO 8601 date/time, project created |
| image | datauri of project icon |
| modified | ISO 8601 date/time, project modified |
| urls | Array of URLs configured for project |
Retrieves a list of resources associated with a project using the Plugin API.
Optional Fields
| Field | Description |
|---|---|
| assets | array of assets, HTML only |
| content | page/file contents |
| created | ISO 8601 date/time, crawled resource |
| headers | HTTP headers |
| links | array of outlinks, HTML only |
| modified | ISO 8601 date/time, resource modified |
| name | page/file name |
| norobots | crawler indexable |
| origin | forwarding URL, if applicable |
| size | size in bytes |
| status | HTTP status code |
| time | request time, in millis |
| type | resource type, html, pdf, image, etc. |
Retrieves a list of crawls using the Plugin API.
Optional Fields
| Field | Description |
|---|---|
| created | ISO 8601 date/time, crawl created |
| modified | ISO 8601 date/time, crawl modified |
| report | Crawl details as JSON |
| time | Crawl time in millis |
MPL 2.0. Earlier releases (0.18 and prior) additionally bundled JavaScript to TypeScript ports within ./src/lib — Typo.js (Modified BSD License) and Snowball.js (MPL 1.1) — which remain under their original licenses where they appear in release history.