Implementation details for GovCheck

Now that GovCheck.net has gone live and is doing relatively well, I think it's time to discuss how the site was built. The driving force behind all the technologies being used in the production of this site is the Python programming language. None of the code that I've written has been written in a language or using libraries written in a language other than Python. With that said, here's a list of the various libraries being used:

• BeautifulSoup - Used in the scraping code. It is truly one of the most intuitive and easy to use libraries I have used (with maybe the exception of the web framework being used).
• SQLAlchemy - The ORM used to commit data being collected by the scraping code into the DB.
• Elixir - This is a declarative layer on top of SQLAlchemy which allows me to have a similar feel to the DB layer in my scraping code as in my web app code.
• Django - This is the Web Framework being used. I won't be the first one to say that it is *the best* web framework I have used - hands down. In addition to the Django Core as well as the contrib apps, I am using multiple third party apps:
  • django-registration - The user registration process on the site is handled by this app.
  • django-tagging - This app is currently used to allow users to assign a custom tag to petitions. I imagine that this app will find many more uses as the site grows.
  • django-pagination - This app handles the pagination on any pages with pagination.
  • I have also developed a couple of my own third party applications (which serve various functions - such as the petitions functionality), some of which I plan to open source soon.
    
• pygooglechart - This is a library that allows users to generate maps using the Google Charts API. It is used to generate the maps throughout the site.
• jQuery - The JavaScript library being used on GovCheck.net. As their website says - "The write less, do more JavaScript library". I'll also add - the best JS library out there.
  

As for the deployment, the Operating System being used is Ubuntu 8.04 Server edition on a 256MB slice. The webserver being used is nginx which is passing on requests to FastCGI instances running in the background. The database server is PostgreSQL. As of now, none of the pages are being cached, but I plan to add that functionality soon using memcached. Finally, all the media files are being served using Amazon Web Services (via a CloudFront distribution). Given this setup, I've got room for approximately 30 req/s on some of the heavier pages. I imagine that this no. will go up significantly when I add memcached to the mix.

So there you have it - the software being used at GovCheck and the servers being used to serve the site.

Inaugaral Post

Welcome to GovCheck!! This is a site where you can find information about Indian Members of Parliment (Lok and Rajya Sabhas). The data, as it stands now, allows you to view MP's attendance and debate records, list of states in the Union, list of parties that have at-least one representative in either of the Houses as well as a list of Contituencies within the country, and much more. All of this data is accumulated from three sites (as of now) - the Lok Sabha, Rajya Sabha and the Election Comission of India sites .

This is the data dispersal part. The site also features ways for people to interact with each other and possibly with their representatives. This can be done (for now) using the Petitions feature on the site, which allows users to create petitions and target them to specific MP's, States, Parties, Constituencies or any specify a custom tag to describe the nature of their petition. Petitioners can then invite their friends and family to go sign said petition. The more signatures a petition has, the more weight it will carry - while will hopefully force officials to take notice.

However, as insinuated above, there is lot's more to come. I will outline in future posts what my vision for the site is, but, for now, suffice it to say that the data sources will grow vastly as will the interaction features.

I hope you enjoy using this service, and would love to hear from you with any feedback, comments or questions you might have.