Bot Database

Vendor Security & AI Research Tools

Security companies use bots for looking at web platform stacks, vulnerabilities and software utilisation metrics. These bots can discover a surprising amount of detail on the tech stack. Obviously front-end components, widgets, Content Management platforms etc. are the easiest to detect, but it doesn’t stop there. Infrastructure details, the CDN, web servers, cloud provider, e-commerce platform, operating system, WAF versions are all accessible with the right tools. This whole area is simply a two-edged sword. On the one hand research tools that assess, find and quantitate vulnerabilities are vital and necessary. On the other hand, do you really want this data in the public domain?

Vendor

Bot Service

Recommendation

Description

logotypelogotype

Keyfactor

Command

Recommended

Not recommended

Public Key Infrastructure (PKI) cloud based CSS certificate management platform

logotypelogotype

IBM

ScoutJet

Recommended

Not recommended

ScoutJet is the web crawler for IBM Watson. IBM is developing next generation search technology, and kindly request that you permit ScoutJet access to your site so that we may refine our relevance algorithms with the broadest variety of content available from the Internet

logotypelogotype

IBM

oBot

Recommended

Not recommended

oBot is the web crawling bot of the Content Security Division of IBM Germany Research & Development GmbH. It crawls to be able to categorize your site into the IBM X-Force Exchange database.

logotypelogotype

Hemholtz Center for Information Security

CISPA Vunerability Notification

Recommended

Not recommended

CISPA Vulnerability Notification bot is used to scape the Internet for information pertaining to the Hemholtz Center’s cyber security research.

logotypelogotype

GrierForensics

GrierForensics

Recommended

Not recommended

Scraper that respects robots.txt for forensic cyber research for govt and private companies

logotypelogotype

Forcepoint

Websense

Recommended

Not recommended

Vulnerability scanner to detect potential weaknesses

logotypelogotype

Experian

Garlik

Recommended

Not recommended

UK online identity and login security service acquired by Experian, yes that Experian responsible for one of the largest data breaches in history.