Train GraphicClick on the map to explore geographics
 
I need help
FAQ
Emergency
About .
Travel & transport from BBC stories as at 03:55 11 May 2026
 
- British passengers from hantavirus-hit cruise ship isolating in hospital, says UKHSA
- This couple lost £1,000 after their flight was cancelled - here is what to check so you don't
Read about the forum [here].
Register [here] - it's free.
What do I gain from registering? [here]
 05/06/26 - EPF Annual Meeting
20/06/26 - Hastings Diesel at Paignton ?
04/07/26 - Railfuture AGM
09/07/26 - Melksham TUG

On this day
11th May (1911)
Last commercial boat - Thames and Severn summit (link)

Train RunningNo cancellations or delays
Abbreviation pageAcronymns and abbreviations
Stn ComparatorStation Comparator
Rail newsNews Now - live rail news feed
Site Style 1 2 3 4
Next departures • Bristol Temple MeadsBath SpaChippenhamSwindonDidcot ParkwayReadingLondon PaddingtonMelksham
Exeter St DavidsTauntonWestburyTrowbridgeBristol ParkwayCardiff CentralOxfordCheltenham SpaBirmingham New Street
May 11, 2026, 03:56:24 *
Welcome, Guest. Please login or register.

Login with username, password and session length
Forgotten your username or password? - get a reminder
Most recently liked subjects
[88] Decisions made by people who don't even live here and use the ...
[83] 13th December 2026 - Great Western returns to public ownership
[73] New Oxford - Bristol direct service, ongoing developments and ...
[67] On tour, May 2026, Grahame
[36] Outage of Coffee Shop this afternoon, 7.5.2026
[33] Return of The Scotsman
 
News: the Great Western Coffee Shop ... keeping you up to date with travel around the South West
 
   Home   Help Search Calendar Login Register  
Poll
Question: Question - which search engines have YOU used which have directed you to the Coffee Shop?  (Voting closed: August 28, 2025, 18:59:03)
Google - 15 (46.9%)
Alexa - 0 (0%)
Bing - 2 (6.3%)
Yahoo! - 1 (3.1%)
Yandex - 0 (0%)
Baidu - 0 (0%)
DuckDuckGo - 5 (15.6%)
Ecosia - 0 (0%)
Some other engine (please follow up to say which) - 2 (6.3%)
I never use a search engine - 3 (9.4%)
I have never been directed to the Coffee Shop - 4 (12.5%)
Total Voters: 22

Pages: [1] 2
  Print  
Author Topic: Coffee shop - which search engines find us, and which do you use?  (Read 5670 times)
grahame
Administrator
Hero Member
*****
Posts: 46877



View Profile WWW Email
« on: August 21, 2025, 07:21:59 »

Question - which search engines have YOU used which have directed you to the Coffee Shop?

Background - why I am asking

We have ten times the number of visitors to the Coffee Shop than we had a year ago. "Users Online" is telling us around 2,000 good requests in the last 15 minutes compared to around 200 good requests on a typical look last year. A very high proportion of the additional traffic is from search engine crawlers and other automata, and to a very great extent it's what I had anticiated as we moved from http to https.  But I need to monitor it and ask / tell / force irrellevant search engines to ignore or go light on our pages - for example, Baiduspider and ClaudeBot are discouraged - told not to index us and if they persist are given a quick forbidden response.

Over the 24 hours to 04:30 this morning, almost 300,000 requests were made of our Coffee Shop worker server to the Coffee Shop URLs. 50,000 of those were from Amazon AI to feed Alexa, 20,000 from Bingbot to feed Bing and 10,000 from Googlebot to feed Google.  Those are the best known; in total a dozen different robots have been indexing us with in excess of 500 page requests each.

Comments welcome on where you might expect to find the Coffee Shop but have not done so.
Logged

Coffee Shop Admin, Chair of Melksham Transport User Group, West Wiltshire Rail User Group Committee and TravelWatch SouthWest Board Member
GBM
Transport Scholar
Hero Member
******
Posts: 1960


View Profile Email
« Reply #1 on: August 21, 2025, 08:12:10 »

I can't remember how I initially found the Coffee Shop. (via Chrome ((edit))
After becoming a regular lurker, I signed up and bookmarket it.
When I joined Firstbus, I verbally told our manager I was a member, but with a few changes of depot, many changes of managers, guessing that was lost in the mists of time.
Always posted carefully, but honestly, where it involved bus items; trying not to bring First into disrepute.

Now retired, I still try to post about what I hear and see about Firstbus (still in contact with current working drivers).
« Last Edit: August 21, 2025, 08:54:14 by GBM » Logged
Worcester_Passenger
Transport Scholar
Hero Member
******
Posts: 2377


View Profile
« Reply #2 on: August 21, 2025, 08:39:54 »

I can't remember how I initially found the Coffee Shop.
After becoming a regular lurker, I signed up and bookmarked it.

Same here.

Meanwhile, I use DuckDuckGo.
Logged
Witham Bobby
Transport Scholar
Hero Member
******
Posts: 964



View Profile
« Reply #3 on: August 21, 2025, 08:53:19 »

Using Brave these days
Logged
grahame
Administrator
Hero Member
*****
Posts: 46877



View Profile WWW Email
« Reply #4 on: August 21, 2025, 09:03:32 »

Using Brave these days

Interesting ...

Quote
Brave Search Crawler

Like any other search engine, Brave search has a crawler to discover new pages and index their content.

The Brave Search crawler does not advertise a differentiated user agent because we must avoid discrimination from websites that allow only Google to crawl them. However, if a domain or page is not crawlable by Googlebot, then Brave Search’s bot will not crawl it either.

My highlighting.   No - they are failing to say who they are - no must about that - it's their choice and as a content provider, perhaps it should be our right at the Coffee Shop to chose who indexes us. As written, this description of the Brave Search Crawler suggests they are intentionally muddying the waters to index us (and use our resources and bandwidth in doing to so) in a misleading way.
Logged

Coffee Shop Admin, Chair of Melksham Transport User Group, West Wiltshire Rail User Group Committee and TravelWatch SouthWest Board Member
Chris from Nailsea
Administrator
Hero Member
*****
Posts: 21879



View Profile Email
« Reply #5 on: August 21, 2025, 12:35:03 »

I registered as a member of the Coffee Shop forum on 23 December 2007, at 00:12:25.  I remember doing so, as I had become increasingly frustrated at the constant delays I had experienced getting home from Bristol on the train, missing our child's early swimming lessons, for example.

However, I have no recollection at all as to how I found this particular forum on the internet to vent my frustrations.  You were all unlucky.  Grin

My current search engine of choice is Google.

CfN (Chris from Nailsea, an administrator on this forum)Wink


Edit to post: I think it may have been MSN, back in December 2007.  Undecided
« Last Edit: September 01, 2025, 21:42:17 by Chris from Nailsea » Logged

William Huskisson MP (Member of Parliament, or Mile Post (a method of measuring the railway in miles and chains from a starting point - usually London), depending on context) was the first person to be killed by a train while crossing the tracks, in 1830.  Many more have died in the same way since then.  Don't take a chance: Stop, Look, Listen.

"Level crossings are safe, unless they are used in an unsafe manner."  Discuss.
grahame
Administrator
Hero Member
*****
Posts: 46877



View Profile WWW Email
« Reply #6 on: August 21, 2025, 12:55:52 »

No utter shocks in the early results - but some fascinating data which is helping inform me as to where all the access are coming from, which are effective in helping us, and which might be a waste of bandwidth / data harvesting for no discernable reason.   Much appreciated, keep comments coming.
Logged

Coffee Shop Admin, Chair of Melksham Transport User Group, West Wiltshire Rail User Group Committee and TravelWatch SouthWest Board Member
Witham Bobby
Transport Scholar
Hero Member
******
Posts: 964



View Profile
« Reply #7 on: August 21, 2025, 13:58:31 »

Using Brave these days

Interesting ...

Quote
Brave Search Crawler

Like any other search engine, Brave search has a crawler to discover new pages and index their content.

The Brave Search crawler does not advertise a differentiated user agent because we must avoid discrimination from websites that allow only Google to crawl them. However, if a domain or page is not crawlable by Googlebot, then Brave Search’s bot will not crawl it either.

My highlighting.   No - they are failing to say who they are - no must about that - it's their choice and as a content provider, perhaps it should be our right at the Coffee Shop to chose who indexes us. As written, this description of the Brave Search Crawler suggests they are intentionally muddying the waters to index us (and use our resources and bandwidth in doing to so) in a misleading way.

I don't think I used Brave to find the Coffee Shop.  That was many moons ago and probably involved either Firefox or the old MS Internet Explorer

Coffee Shop is bookmarked on the Brave browser window, however.
Logged
eXPassenger
Transport Scholar
Hero Member
******
Posts: 658


View Profile
« Reply #8 on: August 21, 2025, 18:34:26 »

I found the Coffee Shop when using Google to look for details of the Reading Station rebuild.  I was then a lurker and later became a member.
Logged
Clan Line
Hero Member
*****
Posts: 1128



View Profile
« Reply #9 on: August 21, 2025, 19:49:43 »

Am a confirmed DuckDuckGo user now. Can't remember how/when I found the Coffee shop - perhaps Google was the only one then !
Logged
Timmer
Transport Scholar
Hero Member
******
Posts: 6862


View Profile
« Reply #10 on: August 22, 2025, 07:11:34 »

I used Google back in 2006 to look for content on when FGW (First Great Western) shortened all their fleet of 158s from 3 coaches to 2 and the chaos that ensued on the Cardiff to Portsmouth line as a result and I’ve been here ever since.
Logged
grahame
Administrator
Hero Member
*****
Posts: 46877



View Profile WWW Email
« Reply #11 on: August 29, 2025, 08:46:23 »

15 out of 22 of you (who voted) have been directed to the Coffee Shop from Google, 5 of you from DuckDuckGo, 2 from another search engine, 2 from Bing. And one from Yahoo.  4 of you have never been directed to the Coffee Shop by a search engine, and three of you never even use a search engine.

Thank you for these inputs.

The Coffee Shop servers get a huge amount of crawling traffic from search engines and other spiders and they have an impact on performance.   Part of the monitoring I do in the background is to see where our resources are going, and to ask some of these spiders to go easy, and if necessary to take action to force them to back off.

I am reading the inputs in association with general search engine popularity surveys,  and for the UK (United Kingdom) Google is even more dominant in general than would be suggested by our survey. Amazingly, the load from the "Googlebot" on our servers is very low and  I would encourage it more rather than less if I could.  DuckDuckGo doesn't seem to be notably indexing us directly - and they do say they use other sources too.

With other crawlers, I take a look at their crawler descriptions - why they want the information, what they are doing with it, whether that's a use relevant to the Coffee Shop and how much of an impact they're having on our server performance.  In my view, there is strength in supporting some AI crawlers to help inform them into the future, and in contrast we should really not impact our user base because of aggressive crawling to inform search engine users who are in countries far, far away.   Those crawlers run by companies who sell onwards services such as SEO (Search Engine Optimisation - web site work to make your site more visible) optimisation or plagiarism reporting but do not make any significant services available without people paying them (for the data they have trawled for free) are typically told to go away if their traffic levels reach a level on our servers that I notice.

The change from http to https last December adds a shell of respectable security to our content, and - as  I would have expected (and indeed hoped) there has been hugely increased indexing activity since. I am somewhat disappointed that the increased activity has not extended to valid new member applications - many reasons and under discussion Behind the Counter, some probably under our control and others not.    The survey helps inform that - only tangentially though; it's rather like asking people on the train why they don't use the train more when really you should be asking people in the street ...
Logged

Coffee Shop Admin, Chair of Melksham Transport User Group, West Wiltshire Rail User Group Committee and TravelWatch SouthWest Board Member
Mark A
Global Moderator
Hero Member
*****
Posts: 2708


View Profile
« Reply #12 on: August 29, 2025, 19:21:20 »

A bit of a read on the woes caused by AI web site crawlers, from the Register.

Mark

https://www.theregister.com/2025/08/29/ai_web_crawlers_are_destroying/
Logged
grahame
Administrator
Hero Member
*****
Posts: 46877



View Profile WWW Email
« Reply #13 on: August 29, 2025, 19:51:08 »

A bit of a read on the woes caused by AI web site crawlers, from the Register.

Mark

https://www.theregister.com/2025/08/29/ai_web_crawlers_are_destroying/


The full headline is "AI web crawlers are destroying websites in their never-ending hunger for any and all content" ... dramatically put and if I hadn't put certain measures in place, our servers would be swamped - and it's an ongoing "game".  Thank goodness I can jump on and very quickly label visitors that match a certain profile as "naughty_boys" and then do a "deny from naughty_boys" and there are some things visitors can do that get themselves automatically added to that group.

The Coffee Shoo IS on shared servers as described in that article, but shared with "my" other sites so if something else such as https://www.twhc.org.uk/person.html - the Well House Collection with records for 52,000 former Melksham residents - gets hit, I am responsible there too - and indeed my server tool are across all sites on the server.   But these days, 4 requests out of 5 are to the Coffee Shop.

Logged

Coffee Shop Admin, Chair of Melksham Transport User Group, West Wiltshire Rail User Group Committee and TravelWatch SouthWest Board Member
Richard Fairhurst
Global Moderator
Hero Member
*****
Posts: 1368


View Profile Email
« Reply #14 on: September 01, 2025, 21:31:58 »

A late nomination for Kagi, which I've been using happily for a couple of years now. It's like Google used to be before it decided it was cleverer than you...
Logged
Do you have something you would like to add to this thread, or would you like to raise a new question at the Coffee Shop? Please [register] (it is free) if you have not done so before, or login (at the top of this page) if you already have an account - we would love to read what you have to say!

You can find out more about how this forum works [here] - that will link you to a copy of the forum agreement that you can read before you join, and tell you very much more about how we operate. We are an independent forum, provided and run by customers of Great Western Railway, for customers of Great Western Railway and we welcome railway professionals as members too, in either a personal or official capacity. Views expressed in posts are not necessarily the views of the operators of the forum.

As well as posting messages onto existing threads, and starting new subjects, members can communicate with each other through personal messages if they wish. And once members have made a certain number of posts, they will automatically be admitted to the "frequent posters club", where subjects not-for-public-domain are discussed; anything from the occasional rant to meetups we may be having ...

 
Pages: [1] 2
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.2 | SMF © 2006-2007, Simple Machines LLC Valid XHTML 1.0! Valid CSS!
This forum is provided by customers of Great Western Railway (formerly First Great Western), and the views expressed are those of the individual posters concerned. Visit www.gwr.com for the official Great Western Railway website. Please contact the administrators of this site if you feel that the content provided by one of our posters contravenes our posting rules via admin@railcustomer.info. Full legal statement (here).

Jump to top of pageJump to Forum Home Page