20100128

Shorty Awards Audit Part 2: Exception or The Rule?

After discovering previously that @mercola had 64% valid votes (IMHO) where as @DrRachie had 88% valid votes, I wonder if this is an phenomenon is the same in other categories. In other words: is @mercola's valid vote percentage the exception or the rule?


To answer this question, I ran the audit script for 1st and 2nd placers in the #music category. Here is the breakdown for the current leader, @yelyahwilliams:



  • deleted accounts: 449 (13.98%)
  • accounts with 1 tweet: 21 (0.65%)
  • accounts with 2 tweets: 32 (1.00%)
  • other accounts: 2706 (84.27%)
  • total: 3211


There is a discrepancy of 3, from users BenFreemann, ovan10, and YannickBraun. 


Now for @ivetesangalo:

  • deleted accounts: 431 (9.07%)
  • accounts with 1 tweet: 30 (0.63%)
  • accounts with 2 tweet: 92 (1.94%)
  • other accounts: 4190 (88.21%)
  • total: 4750


There is a discrepancy of 7 from andreiaAMO, Brubruna, buguinhaBrito, Erica_nasciment, julianestephany, KEKELZITA, and pedro_PERWAH.


It would seem that @mercola's 64% is an exception not the rule. Granted, I only sampled 4 leaders. I suspect however that when I run the statistics for the leaders in #celebrity, I would see the same results: that the rule is for percentage of valid votes are 80% or higher.


I will update when the #celebrity audit is done. Note that I am auditing those categories where the leaders have vote counts in the thousands. This should reduce anomalies. 


Cheers,
Steve

Labels:


20100127

Shorty Awards Audit

After reading the sordid tale of ballot stuffing via twitter over at Bad Astronomy, I wonder if @mercola has the same "problem". Further, I wanted to know if it was also affecting @DrRachie.


To that end I wrote a python script to "audit" shorty award votes. Given a username, the script will scrape shortyawards.com for voters, and hit their twitter.com profile to generate a file containing 2 columns: username and number of updates. Users with deleted accounts will have -1 updates.


I have ran the script for @mercola, and at the time of data collection (UTC 1100) this is the breakdown of where the votes came from:


The discrepancy of 4 comes from users who somehow managed to have no tweets: I suspect the account was deleted, then recreated. These 4 users were: bugoff48, budsgirl54, tracyaustin, janesperr.


You might wonder why I took an exception to users with 2 tweets. The following screen shots should suffice as an explanation:





I checked at random 10 users with only 2 tweets, and they were all people who created a twitter account for the express purpose of voting in the shorty awards, which is against the rules.


Personally, I would say that only 64% of votes for @mercola are valid. This puts him in the lead still, but only ~300 votes in front.


Feel free to do your own analysis of the data.


I am still running the script for DrRachie, so I will update when that script is done. In case you are wondering why it takes so long, that's because I am been nice and rate limiting my queries :)


Update 1: realised some users were showing up twice. Removed them, recalculated, re-linked data.


Update 2: @DrRachie's data is available! See the following.


OK, here is a break down of where @DrRachie's votes came from:




Again there is a discrepancy, this time of a single user, Superpositional.


Just as I did for @mercola, I checked random accounts with 2 tweets. They all broke the rule. These accounts contained only tweets voting in the shorty awards.


My personal opinion is that 88% of votes for @DrRachie are valid, a percentage much higher than @mercola's.


Again, the data is available for your own analysis.


What should be done about this, I hear you ask. Personally I am happy if @mercola and @DrRachie both have their vote count adjusted accordingly.


Update 3: I am running the same analysis for 1st and 2nd place for #music, to see if the same pattern holds. Those results will be in a new post.


Update 4: I should point out that I am aware both @mercola and @DrRachie received votes in multiple categories. But seeing as how majority of votes are in #health, I feel it would be Too Much Effort to separate the vote out. Though if enough people complain, I will fix it.


Update 5Part 2 has been posted. It explores the question whether 64% valid votes is the exception or the rule.


Cheers,
Steve



Labels:


20100110

An Idea for Personal Domains

Given that we can have purely numeric domains, i.e. 131500.info, why not have domains that map to our mobile phone numbers for personal use?

What's the point? For one, you can give out your public mobile number instead of your website, since it is much easier to communicate numbers than domain names. This can be used to communicate other information that is difficult to convey by voice, e.g. emails, skype names, etc. A simple website at $your_mobile.mob would overcome all these.

This obviously has privacy implications, and as such should be entirely opt-in.

Any such system would need to be regulated, ideally controlled by carriers. A custom TLD like .mob would probably be a good idea too. One must also keep in mind however that such easily predictable domain names will be targeted by spammers.

Cheers,
Steve

Labels:


20091217

Nice Work Australia Computer Society

You came to me during my first year in university, and sold yourself as the paragon of virtue and integrity - the kind organisation I would be foolish not to be associated with if I want to get anywhere in Australia doing software, or "Information Technology" as they call it now days.

I didn't join then, because I didn't have the money. I didn't join later because my career focus shifted away from software. I won't join now ever, because you have sold out.

I am referring to the honorary membership you awarded to none other than one Stephen Conroy, Internet Villain of the Year, 2009.

You had such nice things to say about him too:

“We are very pleased to honour Senator Conroy’s contribution and support of the significance of ICT to the economy and the key role of ICT professionals in Australia’s future,” said Mr Wells.

If I am so inclined as to read between the lines, I can't help but get the feeling you are thanking Mr. Conroy for pushing the Internet filter scheme, and in the process provided jobs for the programmers and technicians involved in the various trials and consultations.

“Senator Conroy has always encouraged the ACS in its role as the independent voice of the ICT profession, welcoming our input to various enquiries and working groups, and regularly attending key ACS events. We are grateful for his on-going support,” Mr Wells said.

I don't think Mr. Conroy is listening to your input, or learning from your events - he continues to believe filtering the Internet is doable, and not a waste of time and resources.

I reject you, Australia Computer Society, as "voice of the ICT profession". Your actions are deplorable and shows a lack of integrity. If I was a member, I would be ashamed.

Cheers,
Steve

Labels: , , , , ,


20090508

Facebook python authentication gateway

Edit: it occurred to me what I have below is the basics of a thin facebook api wrapper. I might make it into one at some point in the future.

If you don't know what this does, you don't need it. Hope this helps some one. Written because pyfacebook is broken, always returns error 100.

FB_API_HOST="api.facebook.com"
FB_API_PATH="/restserver.php"

def get_session(auth_token):
        params={
                "api_key":FB_API_KEY,
                "v":"1.0",
                "auth_token":auth_token,
                "generate_session_secret":1,
                "method":"auth.getSession",
        }

        sorted = params.items()
        sorted.sort(key=lambda x:x[0])

        str_to_hash = ''.join(["%s=%s"%(x[0], x[1]) for x in sorted])
        str_to_hash += FB_API_SECRET

        md5 = hashlib.md5()
        md5.update(str_to_hash)

        sig = md5.hexdigest()

        params["sig"] = sig

        encoded_params = urllib.urlencode(params)
        headers = {
                "Content-type":"application/x-www-form-urlencoded",
        }

        conn = httplib.HTTPConnection(FB_API_HOST)
        conn.request("POST", FB_API_PATH, encoded_params, headers)

        response = conn.getresponse()

        print response.status, response.reason
        return response.read()

This Works For Me when I use it with iphone facebook-connect client:

[FBSession sessionForApplication:myApiKey getSessionProxy:myURL delegate:self];

Cheers,
Steve

Labels: , ,


20081015

Dear Steve Conroy,


You can’t be serious.


You wants to filter all Internet connections now, without the option of opting out?! You want every Australian to be on a list so you can see who is “naughty” and who isn’t?!


You can’t be serious.


You wants to filter all illegal material on the Internet?!


You can’t be serious.


How on earth do you propose to classify the contents of more than 181,000,000 websites (Sept. 2008, Netcraft)!? Assuming a government worker can classify a site as illegal or legal in 1 minute, that person will have to work non-stop every day for 344 years to classify 181,000,000 websites.


You can’t be serious.


How on earth do you propose to even filter that many sites without rendering the Internet useless?!  


You can’t be serious.


How can we be sure the site blocked really is illegal, when we can’t access it!? How do you stop the system from abused? How do you correct the system’s mistakes? Why isn’t any of these described in detail?!


You can’t be serious.


How on earth are you planning on preventing people from circumventing your flitters using encryption?!


You can’t be serious.


Get a fucking clue Steve Conroy. The Internet doesn’t work like that. You have a background in the broadcast industry, and the Internet is nothing like it. You should leave technical decisions to people who know what they are talking about and stop pissing off every Internet literate Australian.

DIAF,
Steve

Labels: , , , , , ,


20080101

YAITM - Yet Another Incompetent Telecommunications Minister

It seems a competent telecommunications minister is hard to come by in Australia. Our new telecommunications minister, Senator Conroy,recently demonstrated that he does not understand a) the Internet; b) freedom of speech; c) the concept of the slippery slope.

Firstly, Senator Conroy’s lack of understanding of the Internet:

Senator Conroy says it will be mandatory for all internet service providers to provide clean feeds, or ISP filtering, to houses and schools that are free of pornography and inappropriate material. - ABC news, 31/12/2007

I would like Senator Conroy to explain how exactly ISPs are going do the above for sites which feature mixed content, such as many forums and discussion boards, youtube, and many other sites where content is primarily generated by users. If a discussion board is white-listed, what happens when inappropriate content is posted by a rouge party? There is no technically feasible way to block content on a per-page basis because of encryption and the difficulty automatically classifying content. For example, how can software tell the difference human anatomy and pornography? Further, the Internet is not just websites. There are non-http traffic such as IRC, bittorrent, newsnet, p2p, MSN, jabber, etc. How are ISPs expected to policy those?

Secondly, freedom of speech, a concept which the Senator does not appear to comprehend:

"If people equate freedom of speech with watching child pornography, then the Rudd-Labor Government is going to disagree." - Senator Conroy

Here Senator Conroy equates freedom of speech with watching child pornography, then declares that since child pornography is evil, freedom of speech must also evil. This technique is called a Strawman Argument. It is at best a misleading way of arguing a point. Freedom of speech in no way justifies nor sanctions the creation or distribution of child pornography. Freedom of speech is not a free ticket to anarchy. I would have imagined a Senator would know this. In addition, Internet filtering you can opt-out of can not combat child pornography. Those who are going to look at child pornography are either going to a) subvert the system; or b) opt-out of it. Despite the fact Senator Conroy is using child pornography as a justification for censoring the Internet, it will do nothing whatsoever to curtail child pornography.

Finally we come to the slippery slope which Senator Conroy is happy to slide down:

"Labor makes no apologies to those that argue that any regulation of the internet is like going down the Chinese road," - Senator Conroy

If Labor is making no apologies for going down the Chinese road with regards to Internet filtering, what else is Labor not going to make apologies for? Will Labor also make no apologies to those who argue that governments which do not listen to their civil liberty groups’ concerns is heading down the despot road?

The Senator is, despite his failings, a politician. He is not without his tricks. He seeks to placate those of us concerned with our civil liberties both now and in the future by allowing us to opt-out. Except opting-out means our names end up on a list somewhere, a list some people will interpret as a list of naughty-people who look at naughty-things. Further, it creates a state of affairs where the average citizen only has access to parts of the Internet allowed by the Government. In North Korea a similar state of affairs exists, bought on by decades of suppressing free press and communication. It has led to the citizen of North Korea to regard their oppressed and dreary lives in a third world country the height of human civilisation, a paradise on Earth. Are citizens of Australia so trusting of the government that we will accepted censorship without seeing what is being censored first?

All in all, Senator Conroy is Yet Another Incompetent Telecommunications Minister. I wish that just for once the Telecommunications Minister has a degree in engineering. Then perhaps he or she will understand the futility of attempting to censor the Internet.


Cheers,
Steve

Labels: , , , , , ,


This page is powered by Blogger. Isn't yours?