Insights Delimited By Time

A couple of days ago I started learning python. As I go through the experience of learning another language I want to take a step back and talk about how a person who has siginificant experience in other programming languages might approach learning a new language.

Like most folks in the community I too started out serious development with Java and over the years learnt and unlearnt several techniques with this language. And as a consequence the way I write code today is an outcome of several influences I have had and preferences/styles I have ended up cultivating. For example my increased affinity for unit testing. I tend to follow it very religiously when it comes to doing something non trivial. In fact even when doing something trivial instead of a sysout I prefer an assert statement.

When encountered with a new language, what approach should a seasoned programmer use be productive asap?
IMO what seems reasonable is taking oneself through a breadth first tour of the language -

• Starting with understanding the typing/type system (strong/weak, static/dynamic) style of the language.Play with the shell if any. Use it heavily to check “if this code works”
• Understand how libraries are managed (installed/uninstalled/imported into your code). Explore availability of package managers etc. And as far as libraries themselves go my preference is to start with simple IO followed by some network IO and then some XML parsing and database connectivity.
• Start with your first program doing something non trivial. For example it could be parsing an rss feed. This can turn into an exhaustive exercise. Sucessfully finishing this task means you have learnt how to use an xml parsing library, designed a set of collaborating classes/functions/modules or whatever constructs the language supports and have tested it. In this way you even learnt how to unit test your programs using XUnit for that language.

One another aspect that I have found worth studying is the meta programming and reflection constructs that the language might have to offer. This often helps you write code that is quite terse and often helps you acquire a higer level of expertise/mastery of the language. Though I reserve this experimentation for a later stage in the learning process I think its necessary to do it and more importantly try it out on your starter program you have built thus far.

The objective of this exercise is to get to a stage where you can be nearly as productive as you are currently in your “technical mother tongue”. While this is one of the ways I would love to know if people have tried other approaches. In particular when you are learning a language that does not support conventional paradigms like OO (like Scheme,Prolog etc) will this approach work?
Arguably if one started with a language like Scheme learning other languages will be easier but unfortunately not every CS curriculum/entry level training program encourages it.

I am not sure how many of us have had this kind of an experience but this happens often to me. Over the last couple of days I have been reading about good performance tuning tips and within myself started wishing that I would get an opportunity to fix one.
And (reminds me of Paulo Coelho in The Alchemist - “The whole Universe conspires to give it to you”) the very next day an opportunity presented itself.

An issue suddenly cropped up with one of our RoR applications. All seemed ok after the first week of going live but as more legacy data was migrated into the new app a gradual increase in reponse time became apparent till it reached a point where the response times were clearly unacceptable.

I began the surgery by looking at the code and found that issue was related to the way some ActiveRecord objects were used to render data on some screens. On one of the screens it looked like a collection (about 2000 rows) of an entity with 6+ associations was being rendered on a screen.
Also while rendering on a grid it looked like the user wanted to see not just the root enitiy’s attributes but also some attibutes of its children and grandchildren.
While normally a single query would be enough (and this was the fix I put in) to pull all the 2000 rows, here in this case a naive usage of ActiveRecord caused it to lazy load most of its associations and consequently about 6 queries were being fired for each row. Hmmm….happens.

While this event turned out be not so technically challenging after all it certainly offers a lesson in technical judgement. The problem is not a new one but nevertheless the most repeated mistake I have seen. And the causes of this are more to do with lack of understanding of how to use the tools you work with and the step motherly treatment we give to SQL.

It the desire to build a good OO application (which is hard enough) without worrying about how it is represented in a Relational schema that has created for us a special class of tools that we call ORMs. Different ORMs solve this problem for you in somewhat different ways.
But after working with a couple of different ORMs I am increasingly begining to feel that they are most useful in transactional use cases and anywhere else where you have a show-only/reporting/search requirements you are better off with PO-SQL.

I understand why someone prefers just working with objects than go through the pain of managing connections, creating prepared statements, going through result sets etc. Yeah its so early 2000-ish to do that. Also most ORMs have now acknowledged the fact that there are situations where the user would like to have complete control over the SQL that needs to be executed.
Hibernate goes so far as to let you map a PL-SQL procedure to an object.
ActiveRecord provides a find_by_sql and the nicest thing about it is you can put any query in there like this

list_of_foos =
Foo.find_by_sql(”select foo.name,foo.age, b.city,c.zip from foo,b,c where…”)

And this list_of_foos contains a collection of Foo objects from which you can retrieve name, age, city and zip as though it were its own attributes.
This often surprises people, especially ones doing rails for the first time. Understandable.
And then there is iBatis which is complete freedom in some sense and yeah I am loving it.
With all these goodies one is more tempted to stay away from SQL and treat SQL as a step child. We are so spoilt that today any ORM that “helps” the “object programmer” from dealing with SQL is “perceived” as “better”.
I think here is where the mindset needs to change. While I agree that for most use cases the SQL that is used is routine, learning to write good SQL should be a priority on a pragmatic programmer’s list. Also be careful when you use ORMs. They are great, help reduce lines of code and thus boost productivity but not every use case of your application needs it, atleast not to be used in same way, especially search and reporting use cases.

‘Nuff said !!. Me off to pester the DBA for an SQL Bootcamp!!

After working on RoR for close to an year I can’t stop applying many of the learnings I gathered while I was in that camp.
On my current codebase I decided to solve the issue of configuring things differently in different environments with some inspiration from Rails.

One insightful way of look at Rails itself is to view it as a domain model of a web application. If you just look at “Web Applications” or even “Applications” itself as the domain some of the entities that can be found in this domain would be Models, Views, Controllers, Scripts, Style sheets, Web Services, Databases and last but not the least Environments. Rails quite elegantly weaves these different entities into a neat “Web Application”.

But in this blog let me talk about the much less discussed about “Environment” object. Since the notion of an Environment and seamless collaboration with the rest of the components is built into Rails it often escapes any discussion.

All of us who have created web apps know that they are deployed in many different physical environs. Apart from the basic dev,test,production environs we sometimes created specialized performance or sometimes even a simple showcase/demo environment etc. More often than not each of these atleast have a different database schema at the very least. And sometimes depending on the complexity may deal stubbed/mocked services, queues, different logging requirements and so on.
With this increased set of dependencies comes the issue of configuring each of them differently while keeping the application code agnostic to the env sometimes primarily to improve testability.
When I saw this issue come up in my codebase I decided to adapt the idea from Rails by creating an SystemEnvironment interface. It has different concrete flavors such as ProductionEnvironment and TestEnvironment (just two in my case) and primarily exposes getters for different configurable properties.
With this in place the next thing is to inject a concrete Environment instance into my services. Here again I resorted to a simple constructor based dependency injection with my tests injecting a TestEnvironment instance and the default constructor injecting a ProductionEnvironment instance.

So lets assume you have an attribute that for some reason has to be obtained through a JNDI lookup when running in a container. Now with the indirection of an Environment we have made the client agnostic to how the attribute it depends on is resolved in different environs. And this leads to a significantly better design with the added benefit of simplifying testability.
The test/dev env might just return a hard code value or read it off a file while only the production env actually makes the JNDI lookup.

Apart from making your system more loosely coupled this approach also centralizes configuration and prepares the ground for a more elaborate configuration management implementation such as using the jakarta commons-configuration library.
Another side benefit I realized was by further aggregating different sets of attributes based on what they controlled.
For example a set of attributes were related to my email environment, some influenced what kind of external services I accessed etc.
Such a refactoring would help greatly just to get a clarity on the external dependencies (aka points of failure) a system might have.

With this mechanism in place, one of the problems I am yet to solve is create a simple mechanism to inject the right environment like the way Rails does it, i.e out of a single configuration file. The other thing is to instruct tomcat to look into the approapriate file and init the env. I understand a dependency injector such as Spring would help but my currrent codebase is not using spring so any other “lighter” ideas are welcome.