Search revolution

In light of recurring stories like this one, I think someone needs to develop the next killer OS project: free search.

I would propose that it would be a system of loosely coupled (ie: decentralized, not owned by any one person) servers which all run the same search application and share data with each other, mirroring the indexes in idle time so that if any one server goes down, not much would be lost. It should also be explicitly designed to not to have the ability to store records or logs of any of the search information.

Maybe someone out there has already started on something similar, I’ll have to look around.

20 Comments

  1. Dan
    Posted January 19, 2006 at 4:56 pm | Permalink

    And who would fund this wonderful system? The government?

    Anyways, Google is fighting it.

  2. Posted January 19, 2006 at 7:56 pm | Permalink

    That’s the whole point - it would be developed by people who want to do it for the good of the world and released open source. You don’t necessarily need funding. It certainly wouldn’t hurt though, as long as it can still be open source.

    You definitely wouldn’t want the government (or Google) to do it, not to mention that they wouldn’t make something like this which did not track the usage and was open source so that everyone could verify that it wasn’t tracking.

    While Google is admirably fighting in this case, it has given plenty of info to the government before in other cases. Also, even though Google is refusing in this case (as they should be), it isn’t hard to imagine the government forcing them to comply. The beauty of this system would be that there would be no logs to be fought over, so you wouldn’t have this issue, and no amount of force could make you turn them over since they wouldn’t exist.

    You could avoid the server maintenance and bandwidth costs by making a lot of the indexing & processing distributed (like Seti@Home) and even the searching of the indexes could be done from many sources simultaneously (like bittorrent, but with lots of big dedicated servers linked together). It might not be as fast Google, but at least it would be more secure.

  3. Dan
    Posted January 19, 2006 at 10:08 pm | Permalink

    Or you could just not worry about every little thing and use Google.

  4. Posted January 19, 2006 at 11:54 pm | Permalink

    Which I do, but there are times when it is prudent to do things in a private manner.

    Waiting until such a situation arises to be sure that there is a system in place to support it is a recipe for disaster, whereas pushing for such a privacy preserving system now (even when you have nothing to “hide”) should send a message that reinforces the fact that the governments ability to “search” you should be as limited as possible, and only apply under specific conditions, not including blanket and/or random dumps of data from Google (which is what they’re asking for now).

    I’ve had this same argument a million times before with people who don’t understand why I suggest that more people should use public key encryption in everyday email.

    There’s a simple explanation: the more people use it, the less the messages that really need it will stand out from the crowd. People will say, “I have nothing to hide”, but then I would ask why they mail things in envelopes instead of just using postcard-style open letters all the time. Just because you would prefer other people not read / view what you’re doing, doesn’t imply anything negative.

  5. nstryker
    Posted January 19, 2006 at 11:55 pm | Permalink

    i for one welcome our new google overlords.

  6. Dan
    Posted January 20, 2006 at 9:48 am | Permalink

    We should also not that other search engines HAVE given into the feds and I wonder which.

  7. Dave Z
    Posted January 26, 2006 at 8:43 am | Permalink

    I can’t resist throwing some fuel to this.

    From a Yahoo column by Harold Maass this morning:

    “BEST COLUMNS OF THE DAY

    Web surfer, protect yourself

    The “brouhaha” between Google and the Justice Department is “a big public relations stunt,” says Adam Penenberg in Slate.com. “

  8. Posted January 26, 2006 at 7:13 pm | Permalink

    Public relations stunt? That doesn’t make any sense.

  9. nstryker
    Posted January 26, 2006 at 10:10 pm | Permalink

    so google is pretending to not cooperate to keep the illegal porn community happy?

  10. Posted January 26, 2006 at 10:47 pm | Permalink

    As far as I can see, Google is lawfully and admirably refusing to volunteer information for which the government has no right or reason to request. The reason it’s a request (as opposed to a forced issue via a warrant or some other means) is that there are specific provisions in the search and seizure laws to forbid fishing expeditions (no names or specific details provided by the government, etc.), and that is exactly what this is.

  11. Dave Z
    Posted January 27, 2006 at 12:21 pm | Permalink

    the government gets to look like a protector of kids, google gets to look like the free speech ” we do have all the records but you can trust us, look how we are fighting the government, so keepign using google..”

  12. Posted January 27, 2006 at 2:47 pm | Permalink

    So you’re saying it’s a conspiracy between the government and Google? I doubt it.

  13. nstryker
    Posted January 28, 2006 at 8:29 am | Permalink

    it’s a conspiracy in the same way that everything that is a conspiracy is: just two juggernaughts fighting for self-preservation.

  14. Posted January 28, 2006 at 5:47 pm | Permalink

    Conspiracy usually involves two (or more) parties working together.

  15. nstryker
    Posted January 28, 2006 at 10:24 pm | Permalink

    i guess my point is that everything that appears to be a conspiracy is really just a bi-product of an overly complex system.

  16. Posted January 28, 2006 at 11:52 pm | Permalink

    Ah

  17. Dave Z
    Posted January 30, 2006 at 8:06 am | Permalink

    think non-concretely about this. Conspiracy - No, political game - yes.

    The clinton justice dept did it too. It’s called politics and intergovernmental diplomacy.

    Not a conspiracy at all but a stupid little circle game where by each party knows when starting out how far this will go……..nowhere.
    I know if I sue someone and the law is on their side then I know pretty much how it will turn out. etc.

    The upstart at the justice dept. decides to try to get the info out of google, under the guise of proctecting kids from child porn, google of course has to resist thisand looks great doing it.
    Lawyers present strong agruments back and forth and the result comes in the middle, anymore that results after the publicity for each’s own purpose is icing on the cake.

    Notice how Every president and every major politician gets sued and they start some investigation to ” get them “. With or without merit - it’s part of the mean game of politics to slander, distract, throw your opponent off track so they won’t get re-elected.
    It’s a big (ugly at times ) game.
    Anyway.

  18. Dave Z
    Posted January 30, 2006 at 8:08 am | Permalink

    anyway, I am getting sucked in to these discussions again, which I don’t want to.

  19. Posted January 30, 2006 at 6:26 pm | Permalink

    I hear what you’re saying, but I don’t see that playing out in this case.

    I think that this government honestly does think that it should be able to do stuff like this, and I think Google is just responding naturally to the fact that they don’t have to give up that kind of information.

    If it went to court, the government would be easily defeated on this, unless they change the laws. Court orders and/or subpoenas for information have to be issued in relationship to specific cases, not just blanket requests for non-specific targets. This is why the went the route of “requesting” the information rather than going through legal channels.

    This is the same thing for the wiretap controversy recently. The NSA has always been able to do wiretaps when they actually needed to, after getting the appropriate court orders, which were basically rubber stamps, as there have only ever been two or three denied requests out of thousands. If the timing was critical, they could even go back and get the court orders afterwards, assuming they were justified.

    The problem was that now they wanted to tap any number of unnamed people at their discretion, without providing any justification or explanation for their actions, either before or after the fact. This is why this order was a crime, and in turn why they attempted to keep it secret (including the president outright lying about it, specifically). Coming up with the pitiful excuse of the violation of the law being justified in the name of “fighting terrorism” is even more ridiculous, because if you accept that then the same excuse could be applied to pretty much any action the government decides to take.

    Note: I’m not trying to suck you in with this, so don’t comment if you don’t want to.

  20. Dave Z
    Posted January 31, 2006 at 10:26 am | Permalink

    ha ha, very slick with the wiretap seque, I am impressed, even for a subject changer like myself… that was good.
    I am not getting into the wiretap discussion.

    Power vs power as a free flowing dynamic within the government finding that the checks and balances do work for the most part, I think.

    I am sure there are those agressive players within justice that want more than they can have, I saw it in the clinton admin. also, using the highway fed. funds as a power tool to make states do want they wanted or threaten them into submission. The ACLU ( the osama of lawsuits ) does it all the time, ‘if you dont take that cross off the LA city logo we will sue.’ The city may or may not win but usually don’t wnat to spend the money defending itself.

    I remember in the last election one of the political operatives said he thought about starting a rumor about a candidate having sex with sheep. He knew it wasn’t true but just wanted to hear the person deny it on the news, for the tainting it would do.

    I am cynical enough to think they ( the government ) play games like let’s sue and counter sue just to get in the NEWS. A little like Lindsey Lohan always having some drama and getting in yahoo alot. Today it was stitches on her leg. ARGGgg, I might have to switch to having google as my home page.

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*