Forum Privacy, Search Engine Indexing, & Visibility

Topics about or relating to the forum itself and how it's run.
Post Reply
User avatar
Allan
Posts: 933
Joined: 30 Aug 2003 20:44
Location: Victoria BC

Forum Privacy, Search Engine Indexing, & Visibility

Post by Allan » 29 Nov 2015 12:45

I've been meaning to write this post for some time, but other priorities have seemed more important. With participation at all time low, I've felt somewhat empowered in my position to make some low-level decisions about the architecture and features of this forum without discussion, despite being conflicted about them. Bad Allan! Here I'd like to discuss a couple of these decisions that have been difficult and hopefully solicit some feedback. I'm compressing my argument severely in the interest of brevity.

Despite the fact that the concept of personal privacy rights is a recent invention (past ~150 years or so), I think it is a very valuable and useful construct for society, and that people have a right to basic privacy. I also think that capitalist market forces (note that I don't use this term in the pejorative) are driving technological advances that allow businesses to increasingly rely on invasive tactics that violate basic privacy in exchange for so-called "free" services. I absolutely can see the benefit that a company such as Google can provide to a person when armed with such big data (it's truly fantastic), but I'm not alone in wondering aloud about the consequences; these benefits must also come at some cost, and while it may be that an individual can personally decide that they are willing to make the trade off between their privacy and the use of, say, gmail, increasingly, this decision effects everyone you communicate with over that medium.

Are you comfortable trading my privacy for your "free" email too? It's a valid response to say that you're comfortable with that. But doesn't that make you think a bit deeper about it? You accept gmails ads in exchange for sending/receiving, but by sending me an email, Google is also (likely) building a profile about me with the contents of our discussion, whether I want that or not. If it's not Google compiling this information, there's some middleman (ahem Facebook ahem) out there slurping up our data and using it to sell us stuff. This practise may turn out to be innocuous; it probably won't lead to the downfall of civilization, but at very least it's interesting food for thought.

Personally, while I used to wholeheartedly endorse Google services (and still do in some circumstances), I've come around to wanting to opt-out of being tracked by advertisers in general. Several high-profile cases in recent years (Snowden et al) have bolstered my opinions on this (adding to legit evidence that tracking & snooping isn't some wild malicious conspiracy), and while I certainly don't see Google as "evil" or subscribe to conspiracies of that nature, I think it absolutely wise to for people to retain control over their online presence and exercise the ability to opt-out of tracking, if that's what they want.

I want to opt-out to as much a degree as is truly practical at this point, and, as thus, I am forced to run tools that hide my presence on the web to something I feel more comfortable with; I'm not on Tor running through VPSes or anything, but I take some reasonable steps to reduce my footprint on the web and give advertisers less of a surface to work from; I want my "profile" to be small and almost useless in selling me anything. I don't think this is being irrational.

To the point: There are a couple of things I've done to modified in recent years that align with my feelings on this matter. In some cases, this has probably led to a less convenient or useable website for some people. It's not a tradeoff I've felt very comfortable in making, hence I've finally made this post to see if someone strenuously objects to my handling of things.

Some of the things I've done to make modified more private:

robots.txt denies all search engines. Some search engines just ignore this, but most respect it. This means that we don't show up in Google for searches for something like "footbag." Before you freak out: this forum has never ranked on any search engine for generic "footbag/hacky sack" terms, so it's not as if me doing this has decreased our search engine "exposure." Seriously: I've been involved with basically the only white-hat, ethical SEO (search engine optimization (a BS industry BTW)) in footbag, having managed freedomfootbags.com for many years, as well as the worldfootbag.com redesign I did back in 201... something. modified.in (or .ca before it) has never ranked on the first three pages for anything a new player would search for. You'll just have to trust me on this.

I did this primarily because I don't think that our discussions should be easily publicly accessible. I don't think you should be able to search for "allan haggett" and get links to my posts here. I think quite a few of us feel the same. I think making the forums unreadable unless you're logged in is a step too far, but doing this gives us a reasonable amount of privacy, while still being "public."

Unfortunately, as all search engines are denied, "google site search" no longer works for modified at all. The built-in search engine works--kind of--but really sort of sucks. It is in the middle of my priorities list to fix somehow (probably by fixing the UI for built in search, but I don't know). I think Google used to support what was basically a private "site search" (search appliance/box?) but I think maybe that relied on public crawling too. This is certainly the most egregious consequence to blocking public indexing, and, as I said, I'm not sure how to fix it without allowing public crawls that I'd like to avoid.

There is no "tracking" on the site and never has been. Basic logging still happens server-side, but I'm actually considering turning most of that off. I've not looked at the stats in over a year. For basic stats analyses, I used to use Google Analytics for a number of years, and for about a year I used my own analytics server. But for a number of reasons, I've simply turned them off. The pages load faster now :) If we ever actually need to know the stats, we can parse the raw log files or turn js-based tracking back on.

As a consequence of the "no trackers" policy, I've decided not to implement "oauth" social login plugins as they are essentially the ultimate trojan horse for tracking.

Finally, I implemented TLS secure connections (httpS://modified.in/footbag) so that all communication between your browser and the website is encrypted. There is basically no downside to this, except for the site to potentially load a bit slower in certain circumstances (undetectably so in my testing).

In closing, as we can see, the big decision I've made here is to block search engines. Has this been a massive mistake that needs to be fixed? Or should we stay the course and stay as private as possible?

Kylescook
BSOS Beast
Posts: 477
Joined: 14 Jun 2014 06:14
Location: St.Louis, Missouri

Re: Forum Privacy, Search Engine Indexing, & Visibility

Post by Kylescook » 08 Dec 2015 06:45

I don't have well thought out opinions on this topic. I do find privacy to be important. I like that I can post pretty personal things in my blog without worry that a quick search of my name online will bring them up. I think your'e handling the situation well.
Kyle Cook
St.Louis
My Footblog
My Challenges

User avatar
Tjuggles
Flower Child
Posts: 1746
Joined: 27 Mar 2005 20:03
Location: Wellington, NZ
Contact:

Re: Forum Privacy, Search Engine Indexing, & Visibility

Post by Tjuggles » 08 Dec 2015 11:11

I agree with Kyle. I think encryption is the way to go. Thanks Allan!
TJ Boutorwick

"You can do anything" -Greg Nelson

User avatar
isirc10
Fearless
Posts: 572
Joined: 18 Jun 2012 09:13
Location: Penn State

Re: Forum Privacy, Search Engine Indexing, & Visibility

Post by isirc10 » 18 Dec 2015 06:57

I like that this forum is somewhat out of the way. Though I don't post anything bad, there is a lot of information that the world doesn't necessarily need to see about me. I really appreciate the work you're doing to keep things that way. Facebook and footbag.org are good for publicity - the forum is our nice corner of the internet where we all know each other.
Ivan Iakimenko

Maker of I^2 Footbags!

boyle
Post Master General
Posts: 3146
Joined: 19 Sep 2007 23:42
Location: Canberra, Australia
Contact:

Re: Forum Privacy, Search Engine Indexing, & Visibility

Post by boyle » 18 Dec 2015 12:29

I think this is really good work, this is like a little club where you go in, sign in at the door and have a chat with people who are on a similar wavelength. There are enough options to find out about footbag, this forum is more for people that already know.

User avatar
Allan
Posts: 933
Joined: 30 Aug 2003 20:44
Location: Victoria BC

Re: Forum Privacy, Search Engine Indexing, & Visibility

Post by Allan » 18 Dec 2015 17:46

Thanks folks. I appreciate the words :)

Post Reply