16 October, 2011

The 10 Commandments of Efficient Coding.

I was recently asked for advise on how the manage a development project and decided to make it into a 10 commandments of efficient coding. This is ideal for any team size and if followed diligently leads to efficiently written code. Have any more that you would add to the list? Feel free to indicate yours in the comments below.
  1.  Thou shalt spend at least 50% of ones time thinking about a problem before actually beginning any code. Conceptual design is critical to properly grounding the code in the ballpark of the problem being solved such that the solution spans the problem space as much as possible. Not attending to this stage could lead to spectacularly bad choices of solution for the scope of a problem. 
  2. Thou shalt document everything. Every single thought on the development, ideas of implementation should be written down. I keep multiple "to do" lists in .txt files and as well in the memo field entries for my source control tool when I am checking in new code. Documentation is crucial to illuminating the dynamic thought process that informs code and allows forensic analysis in the event of problems to proceed quickly. Inside code, it goes without saying that any part of code that can't have it's function inferred on inspection should have supplemental comments added to do so...this is why comments are formal part of programming languages, use them or get lost in a sea of code mounting that would require developer time jut to figure out what it does before one even embarks on refactoring it.
  3. Thou shalt master object oriented programming. Often the debate is made that one has an equal right to choose object oriented (OO) versus procedural methods when coding but this is nonsense. OO is the default truth of reality, biology is OO, engineering is OO, physics (energy transfer and mediation) is OO and coding in that way ultimately returns efficiency benefits down the line that often can't be seen when coding. The basic tenet of code reuse ultimately enables the developer to apply conservation of energy to the act of solving big problems. The extra time that might be taken to code solutions in OO more than returns itself in much easier extension of the code as new needs emerge down the line. In some areas procedural solutions are all that are needed say for various types of scripting but even these can be embedded into an OO process. For example I use an OO workflow and file version management system that allows me to manage changes to procedural scripts so in a way OO is enabling those scripts to be more efficiently managed and executed by composition inside an OO framework.
  4. Thou shalt socially code as often as possible. Pair programming or extreme programming either through side by side programming of a task or via a visibility workflow system (like action oriented workflow) allows foolish mistakes to be caught as soon as possible. If a developer can get the aid of a buddy to assist in debugging something she's been working on for the last hour to solve the problem in a few seconds with fresh eyes than all those saved hours will add up. The milestones will be hit and more work can be done in less time. Never waste developer time with undo meetings, creative people should be given time to be creative...facilitate their creativity and more work will be done.
  5. Thou shalt always unit test ones code. Never write more than a few lines of code before testing, OO languages like java, c# make this trivial by provide the main() facility for inline testing of new code. One should never code a new method without testing how it works in main() if that method has more than 5 lines of implementation testing should probably follow with each 5 lines of code written. Catching scope errors, caste errors or null exception errors for passed in attributes during the method implementation stage is far less costly than coding everything up and then deploying the code only to find that the client coding bubbles up nasty pathologies that are hard to find. So testing in main() should be reflexive, writing test code for each method that follows one or all of the possible use cases in client code mandatory as one is writing it and one will encounter much less errors at run time in client code and the ones that one does find will likely be sourced in the client code and will not require refactoring and recompiling in the core class hierarchy.
  6. Thou shalt always close class instances that can be opened. Classes with "close()" usually do for a very important reason, to prevent some resource access from leaking memory. Be that resource access to a db via an open socket connection or access to a file on a file system, what is opened must be closed. This should not be done as part of an optimization step AFTER coding is complete it should be a core part of coding. Developers should not get lazy about closing opened resource connections otherwise nasty surprises await once client code starts being written and under massive instance creation key resources cause memory leaks that soon crash apps. and there are probably few bugs harder to track down and fix than memory leak bugs..being diligent with closing open resources can remove a significant time sink for debugging once code is deployed.
  7. Though shalt always use "finally" condition for code that can be catastrophically interrupted. In java and c# some code execution events can be interrupted by catastrophic events (pulled plugs, blown hard drives, dead processor fans) well written code heals from these events either by not implementing the results of a method in non atomic ways (ie. using lots of synchronization and critical sections for code that absolutely MUST run as a unit or not run at all)  or by guaranteeing recovery code is run should something happen. Thus both languages have "try" , "catch" , "finally" operations. Open resources in the body of the "try" should always be closed in the "finally". Interrupting exceptions should always be handled in the "catch"...the compile cost of using this condition is far outweighed by final running code that is robust to these type of failures which by the way are another major way in which memory leaks occur.
  8. Thou shalt always catch and handle exceptions, even if "handle" means "write entry to log". Using exceptions to indicate where code went wrong is a critical aid to debugging, nothing tracks a problem back to it's source better than a well written exception design. Ideally each app. should have it's own set of Exceptions that report app. specific failures and report any such failures to a log file.
  9. Though shalt always log. Every aspect of a running application should be logged to at least a text file, more elaborate means can include email or xml files meant to report to web pages but logging is critical to ensuring the health state of a running application. As a cross cutting concern to the problem solved by the app. logging is often seen by developers as icing but it is indeed the cake, efficient logging allows errors to be trapped and noted immediately and give a time line of their occurrence that can be used to tie problems to recently implemented code or other issues.
  10. Though shalt always optimize and use a profiler when available. Profilers allow a running application to be inspected for the long term inefficiency trends in memory utilization and object creation and destruction that can kill an app. for highly scalable uses (ie all the startups that junior coders out of college are writing in hopes they are the next Google or Facebook). Profilers can identify which objects are being created without being destroyed or closed, can show where possible access conflicts loom in code that accesses common resources and can indicate a trend line for how efficiently an app. will scale over time. Most critically to the needs of a running business...profilers allow developers to see how efficient their code is running and where they can gain performance and resource efficiency by simply optimizing code.Tight code is code that does a lot in a little space and is fast and scalable over instances of processing and memory resource...that is critical to the operating cost of the software and the pricing on any services that it provides. Lower pricing for services comes from efficient code which benefits the consumer who chose your service over the other guy on pricing...thus tight code helps you beat the competition. Efficient code is code that puts more money in your pocket once it is pitted against what the competition is doing. Though shalt aim to always reduce the overall code base while adding new features. A truly efficient general approach to a problem is maximally functional while being minimally complex, in the initial feature creation stage code base size scales nearly directly with feature addition as features may be very different but as code reuse of critical elements increases the cost in terms of code base addition for adding new features should reduce over time not grow if the solution is optimal. Optimizations such as name shortening in core classes and UI templates, refactoring of the same and consolidating of block of code that are reused are critical to achieving tight code that runs fast, scales deeply in memory across Users and over all costs less per server....which ultimately benefits the bottom line.

15 October, 2011

The Return of the Space Cowboy, for Facebook

Because Facebook share widget refused to post this directly I had to "cheat". Apparently the video above contains a reference to the file sharing service "piratebay" in the video header and that prevented the Facebook link widget from allowing the share to embed. The reference is a solicitation to download the album at torrent bay and I agree such solication is wrong....but that is what the poster of the video intended...not what I intend. I just want my friends to listen to "Return of the Space Cowboy" the song I was humming this morning and thought...to find on youtube.

So, in a few seconds of being denied I figured I'd just embed the video in a blog post and link the post to facebook. My friends get to see the video as I intended and all is well with the world...of course I could have found another video reference to the song that didn't include any reference to PirateBay but then the question of "could it be done?" that sprang into mind the moment Facebook denied the link wouldn't have been answered. ;)

More about this particular youtube video though...it's of a kind I've not seen before.  It actually provides a compilation of all the songs in the album along with an on screen set of links to each song in the album! An excellent use of youtube's in video linking capability that may not be too welcome by some music companies that is for sure! Note this video shows total time as 1:06:00 so the compilation is aware of the internal links and I suppose will play them as a set (or one can click on particular song links to play them out of set sequence)..which is pretty cool.

Addition: Apparently the links are scan to time markers in the single video, so clicking on the links scans to the particular song in the full video compilation...some might prefer this better than I originally thought (I thought it would just play the videos).

03 October, 2011

Self Healing infrastructure means the end of a compensation requirement...

In a post I submitted to Facebook regarding the rise of automation and robotics and the fear of mounting unemployment a discussion occurred in the comments and the question below was asked by Bill Davidson:

"There will have to be a revolution in how we "compensate" people. "

In response I wrote the following hypothetical scenario in the vein of this post and this post that I've written that paint the possible future in a story. The gist being here that over time the *need* to compensate will become obsolete since the agents that we would be getting services from won't require payment. We like wise won't *need* to be performing work to get payment the self healing infrastructure that we construct would just provide it for us.

Imagine this scenario:

You go to bed at 11 pm and promptly at 7 am your robotic assistant raps at your door bidding you to wake. The assistant has already prepared breakfast which is waiting hot in your kitchen. You open the bedroom door and find your slippers placed for your use. You head down stairs and sit down and your assistant asks if you'd like coffee or orange juice, you motion which and it pours it for you. After breakfast you are ready for a shower, the robotic assistant had already left the kitchen while you were eating and prepared a bath and towel. You go up and take your shower and as you exit the bathroom the assistant heads in to clean it. After you've gotten dressed you head for the door to go take a walk, you open the door to find 2 of your other robotic assistants at work, one is planting germanium's near your hedges and another is trimming one of two apple trees you have in the front yard. As you head down the path the assistants of many of your neighbors are also at work. Here, one is replacing the screws on a mailbox door...there, another is walking another neighbors dogs, there further off two robotic assistants are helping a new neighbor move in and are carrying in a couch. Just as you are about to walk down the street a vehicle pulls up and a robot gets out and recognizing you, calls at you and indicates that your weekly groceries have arrived. You continue walking and tell it to go ahead and place the groceries on the stoop and notify your assistant to place them inside.

You continue walking down the street and see robots doing things like sweeping the street, watering lawns and repairing traffic lights. For you this is an average day, you don't have a job to do because the Robots do everything...the groceries you were delivered were not "compensated" for but instead were delivered as part of the active infrastructure established by the local community years before. The milk in the milk container was synthesized using custom organisms that are fed the constituent elements and excrete milk as a by product, these organisms were designed not by humans but by other robots which have rapidly advanced the field of synthetic biology. The meat you are delivered is varied in type and all fresh cuts of genetically engineered tissues all grown in factories run by robots, among the cuts you enjoy are lamb, pork , bison and chicken...all of it virtually indistinguishable from killed animals but none of it required the death of any animal to procure. The cells having been gathered using non invasion swabs and used to create stem cells which were then re-differentiated to express entire tissues and organs desired for human consumption.

The vegetables and tubers packed in the robots container also come from various hydroponic and aquaponic and traditional farms...all fielded and managed by robot farmers. The question of how this infrastructure is powered is the only remaining one and it is answered by the many highly efficient solar power panels on homes all over your neighborhood and in fields in the country side. Super efficient graphene and nanotube constructed batteries store excess energy for months of power even during the winter months. All robots run on highly efficient battery packs that are available at pack exchange stations all over, as you continue walking you pass the nearby one and notice a a few robot assistants leaving the location having received fresh packs...others have plugged themselves into outlets available for any use to charge up. One might still wonder though where all the raw materials for this seeming utopia come from, the answer is as one would expect from extraction of natural resources as always except the agents performing the prospecting, analysis, mineralogy, mining, refining and delivery are all robots all fed by the continuous power infrastructure. Where liquid fuels are still utilized they are derived fuels derived from again ...robot designed synthetic organisms designed to produce all manner of chemical or neutralize compounds molecule by molecule as needs demand.

As you continue your walk you smile to yourself as you recall your childhood of 185 years previous. 185 years and you barely look a day beyond 26 as that is about the age when you started taking various genetic upgrades and treatments to stave off senescence. Today you have blue eyes but was born with brown and in your life have had every shade in-between as suited your fancy. You were born in 2023 in the midst of the creation of the many technologies that today you are fully surrounded by and supported by, in those days people actually used to have to work for food, supplies and to keep their homes tidy!! The children that people your neighborhood mostly have no idea what that is like as their every need is catered for them either by their parents or via proxy of their robot assistants but you remember a time when a living self healing infrastructure didn't exist and you thank the march of technology for making it so.