Announcing django-petitions

As promised when GovCheck was launched, I released django-petitions a few days back. This is the Django application that allows user to create/view/sign petitions on GovCheck. You can go ahead and grab it from GitHub.

The Readme file for the project defines how to fetch, install and use the application. django-petitions is released under the BSD License.

If you would like to give some input, find a bug, or would like to contribute to add features to the app, you can always contact me or raise a ticket on GitHub.

Data from the 15'th term of the Lok Sabha

General elections for a new term of the Lok Sabha were held last month, and the new term of the Lok Sabha started on 1'st June, 2009. The list of the MP's for this new term have now been pulled into GovCheck and are available for your viewing pleasure :).

However, as you may notice quiet some data is missing. There are multiple reasons for this. For one thing, the data on the Lok Sabha site is not complete yet (see http://164.100.47.134/newls/Biography.aspx?mpsno=4569 for example) which means that a lot of data on GovCheck will be missing (see http://govcheck.net/mp/tharoordr-shashi/). The Lok Sabha site is being polled everyday though, so as soon as that data is available on their site, it will show up on GovCheck. Data for previous elections is also missing as of now (see http://govcheck.net/constituency/srinagar-jammu-and-kashmir/ for example) - again because statistics for the latest elections haven't been made available by the Election Commission. I will keep an eye out for said data and will have it uploaded as soon as it becomes available.

In the mean time, enjoy your new list of MP's and do let me know if there are any issues of bugs you find.

Statistics matter

For those keeping track, a few new additions to GovCheck.net were rolled out last night.

• Statistics from the previous election for a given constituency were added. What this means is that on a given constituency's page (see http://govcheck.net/repregion/ajmer-rajasthan/ for example) you can now see how many candidates stood for elections, how many people were eligible to vote, how many actually voted and what the winning margin was. See the screenshot for example.

• The other addition is minor, but given the available data, has the potential to be significant. On a given state's page, you can now, in addition to the voting statistics, see the sex ratio of said state too (see http://govcheck.net/state/rajasthan/ for example). The reason I say this has the potential to be more than just this is because all census data for a given state is now being collected. The next step is to display that data in a useful way now.
• Finally, in addition to the Web, News and Video searches, there is now a Twitter search tab (see http://govcheck.net/state/rajasthan/ for example) on all of the State, Party, Constituency and Representative pages. This should help you in getting a more real time understanding of what is being said about the current object.

Additionally the pretty url's trend has been extended. Now all states, constituencies and parties use better urls (see http://govcheck.net/state/22/ vs. http://govcheck.net/state/rajasthan/). As before, older urls should not break - if you have bookmarks, you will be redirected to the new url automatically.

If you have any comments about this change, do let us know.

Getting a broader view

A new feature just added to GovCheck.net let's you get a broader view of the MP, State, Party or Constituency you are currently viewing. On supported MP's pages, you will now see a Google Map of said MP's constituency (see http://govcheck.net/mp/vajpayee-atal-bihari/ for example) . Similarly, on a given constituency page, you will now see said constituencies location (see http://govcheck.net/repregion/449/ for example). Additionally on all the MP, State, Party and Constituency pages, you will now see a box towards the bottom that displays search results (courtesy of Google) from news, web and video sources. We believe that these changes give users a much broader overview of what is happening with a given MP, State, Party of Constituency.

A couple other (not-so-visible) changes have also been rolled out, such as prettier url's for MP pages (see http://govcheck.net/mp/vajpayee-atal-bihari/ vs. http://govcheck.net/mp/417/). An error with the petition creation process has also been fixed.

As always, if you feel like providing any feedback regarding issues you would like fixed or just something you would like to see done differently, you can submit a report or send us an email.

Implementation details for GovCheck

Now that GovCheck.net has gone live and is doing relatively well, I think it's time to discuss how the site was built. The driving force behind all the technologies being used in the production of this site is the Python programming language. None of the code that I've written has been written in a language or using libraries written in a language other than Python. With that said, here's a list of the various libraries being used:

• BeautifulSoup - Used in the scraping code. It is truly one of the most intuitive and easy to use libraries I have used (with maybe the exception of the web framework being used).
• SQLAlchemy - The ORM used to commit data being collected by the scraping code into the DB.
• Elixir - This is a declarative layer on top of SQLAlchemy which allows me to have a similar feel to the DB layer in my scraping code as in my web app code.
• Django - This is the Web Framework being used. I won't be the first one to say that it is *the best* web framework I have used - hands down. In addition to the Django Core as well as the contrib apps, I am using multiple third party apps:
  • django-registration - The user registration process on the site is handled by this app.
  • django-tagging - This app is currently used to allow users to assign a custom tag to petitions. I imagine that this app will find many more uses as the site grows.
  • django-pagination - This app handles the pagination on any pages with pagination.
  • I have also developed a couple of my own third party applications (which serve various functions - such as the petitions functionality), some of which I plan to open source soon.
    
• pygooglechart - This is a library that allows users to generate maps using the Google Charts API. It is used to generate the maps throughout the site.
• jQuery - The JavaScript library being used on GovCheck.net. As their website says - "The write less, do more JavaScript library". I'll also add - the best JS library out there.
  

As for the deployment, the Operating System being used is Ubuntu 8.04 Server edition on a 256MB slice. The webserver being used is nginx which is passing on requests to FastCGI instances running in the background. The database server is PostgreSQL. As of now, none of the pages are being cached, but I plan to add that functionality soon using memcached. Finally, all the media files are being served using Amazon Web Services (via a CloudFront distribution). Given this setup, I've got room for approximately 30 req/s on some of the heavier pages. I imagine that this no. will go up significantly when I add memcached to the mix.

So there you have it - the software being used at GovCheck and the servers being used to serve the site.

Inaugaral Post

Welcome to GovCheck!! This is a site where you can find information about Indian Members of Parliment (Lok and Rajya Sabhas). The data, as it stands now, allows you to view MP's attendance and debate records, list of states in the Union, list of parties that have at-least one representative in either of the Houses as well as a list of Contituencies within the country, and much more. All of this data is accumulated from three sites (as of now) - the Lok Sabha, Rajya Sabha and the Election Comission of India sites .

This is the data dispersal part. The site also features ways for people to interact with each other and possibly with their representatives. This can be done (for now) using the Petitions feature on the site, which allows users to create petitions and target them to specific MP's, States, Parties, Constituencies or any specify a custom tag to describe the nature of their petition. Petitioners can then invite their friends and family to go sign said petition. The more signatures a petition has, the more weight it will carry - while will hopefully force officials to take notice.

However, as insinuated above, there is lot's more to come. I will outline in future posts what my vision for the site is, but, for now, suffice it to say that the data sources will grow vastly as will the interaction features.

I hope you enjoy using this service, and would love to hear from you with any feedback, comments or questions you might have.