Dear Future Developers

Please store as much data as possible. Do not worry about the cost of the extra storage disks. The value in the data will far outweigh the cost of the hardware. Here are some examples of data that could be stored but is typically not.

Start storing data about the order in which pages on your site get visited. Where do visitors most often land, and where do they go from there? Is there a path that leads to visitors becoming customers? Is there a path that leads to visitors leaving? Both would be good to know. Given enough of this data, it would be possible to predict what pages eventually lead to the most customers.

Start storing log information to a database. Some places do this, but far too many do not. As developers, this should get a higher priority. It is never fun to go debug a problem only to find the log file has been overwritten. Setting up a database for this would definitely save on debug time. Plus, the log data could possibly be helpful for determining trends or parts of the system that frequently have issues. It is important to remember that not all bugs produce errors, thus it is important to store all the log data.

Start storing data about the errors that occur and what(screen/page) caused the error. This information is typically stored in log file somewhere. It is too frequently lost after a couple days. It would be much better to store this information in a database for archival purposes. This is closely related to the previous paragraph.

Start storing information about which fields on a form get updated. Then you can notice if users are constantly returning to the same form to update a different field. Maybe the user was unaware that both fields can be updated simultaneously. Rearranging the fields might create a better user experience, and it will decrease the amount of updates hitting the database.

Start storing data about which buttons and links users click. This is not just the pages visited but the actual user actions. A good web analytics program can cover some of this, but why not store all of it yourself. Then you can do with it as you please. It would be great to know for your site what buttons users click the most? Is it the color, location, neither, or both that determine a popular button? What buttons and links never get clicked? How frequently does the same user click each button? If a user continues to come back and click the same button, it may indicate a navigation issue. There are some nice usability enhancements that can be made with this data.

Start storing data that you cannot immediately see as useful. The bigdata movement is continually showing the advantage of having more data. You never know when or for what the data will be useful.

Many of the current NoSQL choices would be good candidates to store the above data. This data will obviously grow very quickly, and speedy inserts are a must. Therefore, a database like MongoDB, Cassandra or Redis might be a good choice.

What other data do you think could be collected? I am sure there are lots of other possibilities. Also, I am going to take myself up on this challenge. I would like to store more information about the software I build.

Sincerely,

Ryan Swanstrom