Categories
Programming

Write Programs for People First, Computers Second

A very important principle to remember while writing code. In some ways, TDD helps in understanding code, since tests are specifications.

Categories
Programming

LINQ-SQL, Contains, Bug

Recently, client had reported a strange bug, which I had never encountered before. The client was editing value of an int column in a gridview. Once the value was changed, the grid displayed this error –

The incoming tabular data stream (TDS) remote procedure call (RPC) protocol stream is incorrect. Too many parameters were provided in this RPC request. The maximum is 2100.

Now, we were not calling any stored procedure. My colleague found the cause of this problem. We were updating an enitity on cell changed event. To update, we were first making a select query like this,
List cs = (from cl in Dataclass.Consumers
where (cl.ID == ID)
&& draUmList.Select(f => f.Order).Contains(cl.ID)
select cl).ToList();

The above query was getting converted into


exec sp_executesql N'SELECT xyx AS [t0]
WHERE (id] IN (@p1, @p2, @p3, @p4, @p5, @p6, @p7, @p8, @p9, @p10, @p11, @p12, @p13, @p14, @p15, @p16, @p17, @p18, @p19, @p20, @p21, @p22, @p23, @p24, @p25, @p26, @p27, @p28, @p29, @p30, @p31, @p32, @p33, @p34, @p35, @p36, @p37, @p38, @p39, @p40, @p41, @p42, @p43, @p44, @p45, @p46, @p47, @p48, @p49, @p50, @p51, @p52, @p53, @p54, @p55, @p56, @p57, @p58, @p59, @p60, @p61, @p62, @p63, @p64, @p65, @p66, @p67, @p68, @p69, @p70, @p71, @p72, @p73, @p74, @p75, @p76, @p77, @p78, @p79, @p80, @p81, @p82, @p83, @p84, @p85, @p86, @p87, @p88, @p89, @p90, @p91, @p92, @p93, @p94, @p95, @p96, @p97, @p98, @p99, @p100, @p101, @p102, @p103, @p104, @p105, @p106, @p107, @p108, @p109, @p110, @p111, @p112, @p113, @p114, @p115, @p116, @p117, @p118, @p119, @p120, @p121, @p122, @p123, @p124, @p125, @p126, @p127, @p128, @p129, @p130, @p131, @p132, @p133, @p134, @p135, @p136, @p137, @p138, @p139, @p140, @p141, @p142, @p143, @p144, @p145, @p146, @p147, @p148, @p149))',N'@p0 uniqueidentifier,@p1 uniqueidentifier,@p2 uniqueidentifier,@p3 uniqueidentifier,@p4 uniqueidentifier,@p5 uniqueidentifier,@p6 uniqueidentifier,@p7 uniqueidentifier,@p8 uniqueidentifier,@p9 uniqueidentifier,@p10 uniqueidentifier,@p11 uniqueidentifier,@p12 uniqueidentifier,@p13 uniqueidentifier,@p14 uniqueidentifier,@p15 uniqueidentifier,@p16 uniqueidentifier,@p17 uniqueidentifier,@p18 uniqueidentifier,@p19 uniqueidentifier,@p20 uniqueidentifier,@p21 uniqueidentifier,@p22 uniqueidentifier,@p23 uniqueidentifier,@p24 uniqueidentifier,@p25 uniqueidentifier,@p26 uniqueidentifier,@p27 uniqueidentifier,@p28 uniqueidentifier,@p29 uniqueidentifier,@p30 uniqueidentifier,@p31 uniqueidentifier,@p32 uniqueidentifier,@p33 uniqueidentifier,@p34 uniqueidentifier,@p35 uniqueidentifier,@p36 uniqueidentifier,@p37 uniqueidentifier,@p38 uniqueidentifier,@p39 uniqueidentifier,@p40 uniqueidentifier,@p41 uniqueidentifier,@p42 uniqueidentifier,@p43 uniqueidentifier,@p44 uniqueidentifier,@p45

So, if there too many instances of the entity, it could easily exceed 2100. So, the solution which replaced the above query, used join query and where clause, which was more efficient.

Categories
Story

Checkmate

Checkmate. And, the game was finally over. The game had lasted for 6 hours. For those 6 hours, Jay was in a different world. He was in a zone, unaffected by the life he lived; the life which was better ignored. The game, was the only thing that mattered. Jay, was feeling a bit disappointed but there was a strange calmness within him. Why so? The game saw both the players getting an advantage at different moments, but were not able to convert into something meaningful. And, then came the endgame, which involves slow maneuvering of the pieces. Jay, enjoyed the process of planning, predicting, being surprised by opponent’s moves, and finally getting out-maneuvered. And, he was ready to play the next game in the tournament. In life everyday is a struggle. Somethings workout, and some don’t. But, if we accept both with the same equanimity, then the journey is joyful.

Categories
Life

Memories

Where do they exist?
In the brain, or in the heart.
Memories.
Many have faded, just gone
Memories.
Many are still alive, can still feel those moments
Memories.
Helps to connect with old friends
Memories.
Make up the life story
Memories.
Joyful moments, sad moments, silent moments
Memories.
Desire to capture, and relive
Memories.
Wish could remember some of the forgotten
Memories.

Categories
Maths

Intuition behind why 2^-1 is 1\2

^ symbol means exponent
What is 2^3 ? It is 8.
What is 2^2 ? It is 4.
What is 2^1? It is 2.
So, we can observe, that, as the exponent decreases from 3 -> 2 -> 1, each result is obtained by dividing by 2 – 8, 4 (8\2), 2 (4\2).
So, what is 2^0? Divide 2\2. And, the answer is 1 !!.
And, so what is 2^-1 ? Divide 1\2.!!
I learned this at Khan Academy
Another way of looking at negative exponents is,

A negative exponent means how many times to divide by the number.

Categories
Maths

Intuition behind why 2^-1 is 1\2

^ symbol means exponent

What is 2^3 ? It is 8.
What is 2^2 ? It is 4.
What is 2^1? It is 2.

So, we can observe, that, as the exponent decreases from 3 -> 2 -> 1, each result is obtained by dividing by 2 – 8, 4 (8\2), 2 (4\2).

So, what is 2^0? Divide 2\2. And, the answer is 1 !!.

And, so what is 2^-1 ? Divide 1\2.!!

I learned this at Khan Academy

Another way of looking at negative exponents is,

A negative exponent means how many times to divide by the number.

Categories
Life

Who am I

I am a human,
I am an individual,
I am a son,
I am a brother,
I am an uncle,
I am a mentor,
I am a husband,
I am a father,
I am a friend,
I am.
Or, Am I.
Is everything, just an illusion?

Categories
Life

Positive Thinking

I got this sms today –

POSITIVE THINKING is not about EXPECTING the Best to happen. But, its about ACCEPTING that whatever happens is Always for the Best

I agree with the sentiment. Whatever will be, will be. Acceptance is the best way to be peaceful. But, it does not mean that we should avoid taking an appropriate action, which each moment, requires from us. Every moment, is full of choices. It is up-to us, to select a choice which reflects our temperament or, we select that choice which we know will be the best for the given situation. The choice we make, may lead to an unpleasant result. But, we should just reflect and move on.

Categories
Django

Django Lessons – #2

I needed to record the IP address of the user as well as the admin for every action completed. And, this IP must show with each log entry. So, I used user_ip=request.META["REMOTE_ADDR"]). But, this did not work on the server. The IP was getting recorded as 127.0.0.1. Why so? The information I got from the webfaction support was that

REMOTE_ADDR is 127.0.0.1 because requests for your site are proxied through our front-end web server, which of course is on the local host.If you want the original IP of the request, then use
request.META[‘HTTP_X_FORWARDED_FOR’].

I did some research, and found about the concept of reverse proxy.

Some links
Forum
Apache

Categories
Programming

Experience Report – From Zero to 500

Every experience teaches something. It is upto us to imbibe the lessons. Sometimes worrying about success and failure takes away the of joy of just doing.

I started working on the current project in September 2009. The technology stack is .net 3.5, Windows Forms, SQL Server 2008 and LINQ-To-SQL. The Project Manager hired me. When I joined the team, the software had been in development for 4-5 months. All of us are from different regions(Middle East,Europe) and work remotely. We communicate by using software like Skype, TeamViewer, and of course, email. We use a wiki for documentation, and a web-based bugtracker. Initially, I was given the task of understanding the software, and writing the documentation. I found this a bit strange. But, it turned out to be a good way to understand the software. Also, I was happy that a wiki was used for the documentation. My many doubts were clarified either by the PM or, the client(end-user). Soon, I found there were many problems –

1) The wiki had all the information related to the system to be developed, and tutorials for the users. But, there were many ambiguities in the system documentation. Also, it was a bit outdated, and not in sync with the state of the software.Most of the domain knowledge was with the PM. And, one of the reasons, he had asked me to do the documentation was to capture that knowledge.

2) The codebase had huge amount of duplication. There were no tests.

3) There was an architecture – 3 logical layers – windowsforms , business layer and data access layer. But, there was lot of logic which was in the forms.

4) All the features which had been implemented were incomplete and had bugs.

5) One of the developers involved had this strategy – he would fix the bug assigned to him and close it.He did not check all the scenarios. So, other bugs cropped up. The software was unstable. I felt very uneasy about this situation.

6) We were using LINQ-to-SQL, but did not have good understanding of this technology. This was creating problems like too many queries getting fired. We realized later when we started facing performance issues.

7) The database was not well designed. There were some columns in some tables not being used. Most of the columns had “allow nulls”. It was not under version control. We had views which were doing lot of calculations.

The PM knew about these issues, and was eager to overcome them. One thing I appreciate about him is his persistence and eagerness to solve problems. We started talking about testing, and he suggested to write functional tests. But, we found that the Devexpress controls did not support UI testing using Microsoft framework. Then, I told him we should start writing unit tests, and he was open to this idea. I also suggested that we use MVP(Model View Presenter), and take all the logic out of forms. He agreed, and asked me to do it on a couple of forms. I started refactoring, and adding tests.

Initially, I was the only one running the tests. Sometimes I found compile errors in the test project which surprised me! I realized that the project has not been added as part of the solution!. Initially, many of the tests were integration tests and so the database was involved. We created a script to create the database and, separate data files(sql) for default data. Then, wrote a script which created the database, build the projects and ran the tests. We used NDBUnit for input data to be used for integration tests. This required creation of xml files. Slowly the test coverage increased, and it started catching some bugs. Then, I started writing tests for bugs too. I realized some things were common on all forms. This, was a eureka! moment, seeing patterns among the mass of duplicate code or, the representation of similar ideas. This led to creation of base view and base presenter. This eliminated a lot of duplication. Soon, we had 2 new members in the team. They were eager to learn and write tests. This helped in speeding up of writing tests. Today, we have about 500+ tests. Also, we started a QA page which listed points to be checked for each module. This page helped everyone, including the QA team.

The state of the software today is –

1) The software is in better stable condition. Most of the business logic is now in good condition, with very few bugs.
2) The user interface has still some issues.

3) We have many tests, but some of them are unreadable, look very ugly. We have have used BDD style of language for some of the tests in given\when\then style.

4) Linq-to-Sql Entities – They had only set\get properties. We had committed
violation of “Law of Demeter” throughout the codebase. Instead of asking Customer.IsYourName(x), we had code (if customer.Name == x). So, we did some refactoring, and added more stuff to the entities like validation rules, moved functions which acted upon “entity data” to entity.

5) A lot of duplication still exists, which can be removed.

6) We have an acceptable performance metrics page for different scenarios. We have tested a lot, and improved the linq-sql queries and the stored procedures. But a lot of work still needs to be done.

7) The software, is currently in testing stage. The users are testing the software with live data for some part of the system.

Some concluding observations –

1) The architecture of a system can evolve. And, an evolving architecture solves the problems faced, so everyone in the team is able to understand the reasoning behind the changes.

2) Some of the decisions related to architecture could have been made early. For example, the application is for multiple users, and these users are located in different locations(countries). So, it seems obvious that some thought should/could have been given – Where will the database be located? Will we need replication to handle some scenarios? Should we develop a desktop or a web application? Also, it seems there is lack of literature for such scenarios. What are the best practices which have succeeded? Or, the way we have approached, taking decisions much later like for replication. In fact, once replication was implemented, we had to change all primary keys to guid from int.

3) A remote set of developers can work well. But, I miss pair-programming. I tried it a couple of times, but due to slowness of broadband, it did not work out.

4) The unit\integration tests gave confidence to the developers as well as the stakeholders. The stakeholders had become jittery, and were reluctant to request changes, since after every change old problems cropped up. But, having regression tests helped to track such bugs and prevented from re-appearing.

5) The documentation of various aspects of the project on the wiki has helped new developers and the QA process. But, the issue is of keeping it in sync with the changes being made to the software. And, I think we are not always in sync.

6) The tests have really helped us. But, I wonder if we are writing too many tests. The test coverage is around 60 %. I feel, since we have remote developers, and also turnover in the team, these tests serve as a good safety net.

7) We need a continuous integration server. The developers are not disciplined enough, and don’t run tests before every commit. This means, if tests are failing then sometimes it goes unnoticed. We deliver software to client and QA every few days. But, building a setup is a manual process, and done by a developer on his local machine. So, sometimes local changes creep into the build.