StackMangoBot

Why StackMangoBot visited your site

StackMango users look for creators, publishers and businesses to work with. When a user’s search turns up your website, for example in web search results or as the website linked from a YouTube channel, StackMangoBot visits it to look for a published contact address. It also visits a website that a user adds to one of their leads by hand.

It reads the page that was found, usually your homepage, and at most 4 contact or about pages linked from it. It looks only for published email addresses, links to your contact and about pages, and your site’s name from its title or metadata. It does not store the rest of the page.

How to recognise it

StackMangoBot sends this User-Agent with every request to your site:

Mozilla/5.0 (compatible; StackMangoBot/1.0; +https://stackmango.com/bot)

Its name in robots.txt is StackMangoBot. The name is matched without regard to case. StackMangoBot does not use fixed IP addresses, so identify it by its User-Agent. Other software can copy a User-Agent: if a crawler using this name ignores your robots.txt, tell us through our contact form.

Control it with robots.txt

StackMangoBot follows the rules for StackMangoBot in your robots.txt file. If there are none, it follows your rules for all crawlers (User-agent: *).

Block it from your whole site

User-agent: StackMangoBot
Disallow: /

Keep it out of part of your site

User-agent: StackMangoBot
Disallow: /members/
Disallow: /checkout/

A group for StackMangoBot replaces your rules for all crawlers as far as StackMangoBot is concerned, so repeat any of those rules that should also apply to it. StackMangoBot keeps its copy of your robots.txt for up to 24 hours, so a change takes effect within a day.

If your site has no robots.txt (for example, it returns 404), StackMangoBot treats the site as open. If the file cannot be read because your server returns an error or asks it to slow down, StackMangoBot treats the whole site as off limits and checks again later. It does not use Crawl-delay: its own limits below keep visits rare.

How it behaves on your site

• It requests at most one page from your site every 24 hours, however many StackMango users’ searches found it.
• It reads at most the first 2 MB of a page, waits at most 15 seconds for a response, and follows at most 5 redirects.
• It reads only web pages. It skips files such as PDFs, images and videos.
• It never signs in, fills in or submits forms, or runs JavaScript.
• If your server answers 403 (Forbidden) or 429 (Too Many Requests), it stays away from your site for 24 hours, or for as long as your Retry-After header asks, up to 7 days.

What happens to the addresses it finds

An address StackMangoBot finds is stored only in the private workspace of the StackMango user whose search found your site, together with the page it was found on and the date. Each user’s results are private to their workspace.

StackMango never sends email to the addresses it finds, and does not sell, rent or publish lists of addresses. If a user decides to contact you, they write to you themselves from their own email account. Our terms of service forbid spam and unlawful or deceptive outreach.

Ask us to stop it for every StackMango user

robots.txt is the quickest way to control StackMangoBot. You can also ask us to:

• stop StackMangoBot visiting your site for every StackMango user, or
• delete an email address from every StackMango workspace and stop it being collected again.

Send your request through our contact form with the website address, such as example.com, or the email address. A site opt-out stops future visits; to remove an address that was already found, include the address too. We may ask you to confirm that the site or address is yours. Our opt-out list stores only a one-way fingerprint (a hash) of each address, not the address itself.