If you operate a store and found this URL in your logs, this page is for you.
gearherd is a spec index for bags. It reads published product data from brand storefronts and normalises it so that a volume, a weight and a set of dimensions can be compared across brands that all describe them differently. It currently covers 484 models across 11 brands.
| /robots.txt | Checked before anything else. If it disallows the path, we skip the store entirely and record why. |
|---|---|
| /products.json | The storefront feed Shopify publishes by default. Roughly one request per brand, nightly. |
| /collections.json | Occasionally, to read how you categorise your own products rather than guessing from titles. |
| Product pages | Once per product, for dimensions your feed does not expose. Cached, so a page is not re-read until the product changes. |
Only endpoints published for anyone. No login, no accounts, no CAPTCHA
solving, and no proxy rotation to get around a block. Requests are
paced, identify themselves, and honour Retry-After.
Facts only — dimensions, weight, price, materials, stock. Marketing copy is not reproduced, and images are referenced at your own URLs rather than rehosted, so they keep serving from your CDN and stop working the moment you remove them. Every model page links back to the product on your site.
robots.txt is honoured, so a Disallow for
/products.json against User-agent: * is
enough and takes effect on the next run. If you would rather just tell
us, write to the address below and we will remove your store and the
data taken from it.
A real person reads this: crawler@gearherd.com. Include your domain and we will confirm what we hold and what we have removed.