29 February, 2008

implementation of guest messaging done ....

...but now comes the testing. I was up late last night testing , or rather getting rid of a bug in the implementation code from last night. Considering the large number of changes I made before testing two hours of bug quashing wasn't bad at all. Though , more efficient code and test practices would have ensured that I performed a unit test after each change , the nature of this addition makes that some what impractical due to the dependencies among the various things that needed to be changed , regardless the bug is gone and now it is time to continue the testing.

I am an avid runner and many times during my runs the solutions to problems I might have been wrestling with or entirely new ideas pop up while I am putting in my miles. During my run today I was thinking of writing a post that shows a bit of the method I use for tracking sub items on my to do list. In that spirit I will copy below the entire to do section for the feature I have been adding over the last 4 days , private guest message support. You will see at once , how many items had to be changed in order to implement the change but note that some of the lines may be some what redundant, they are definitely not examples of my best grammar or punctuation so please disregard that aspect of them. They are a brain dump of the storm of tasks that deluge my mind once I've isolated a method of solution for a given problem. Some of them are obsoleted as during implementation I find more efficient ways of performing the task, which are most likely noted as another task item. It may look like a confused mess but this method has allowed me to ensure that complex coding tasks are completed without missing important subtasks , many of which many not be essential to the overall functionality created but could significantly enhance the User experience or under scaled and loaded conditions significantly ease the developer and administrator experience! Without much adieu here is the list!


GuestPM dashboard

To facilitate guest pm capability for sites I could modify the converse template to support establishing pm’s into the system that tentatively a session between the guest (by ip address) and authenticated Users on the system who have access to a “guest pm dashboard” this dashboard will list incoming guest PM requests and allow authorized users to click on the request to engage the conversation with the guest. This will allow me to license personal help services to other web sites. This was something I thought about doing a while ago and really would be a great “yes we can do that.”


Conversation , might have to modify to allow guest pm’s with hanging end…when a remote guest requests a pm …an unmanaged conversation should be created with its state resident entirely in the participant_queue when a dashboard agent clicks on the waiting guest pm link they complete the participant handshake and can converse. Disable uploads and table file view maintain language options of course (since that’s the secret sauce) Solution: Symmetry

To avoid having to modify conversation it would be nice if I could define a unique characteristic of guest pm’s that can be used to isolate them from standard typed conversations…easy! The conversation id! ..currently negative guest id’s define the guest side , positive user id’s define the user side of conversations of positive value types. To allow guest pm’s if I had zero or negative value conversations I could use them to represent non managed conversations created between users and the guest …these can have hanging ends that can be retrieved for display …simply when a guest makes a request the conversation id being zero or negative means it is a guest request and emit will be modified to create the window, create an unmanaged conversation for it. The guest id can define where the view file will be written for the guest pm session (in a separate folder for guest pm’s) when a valid user accepts a hanging request they can generate the write path of the hanging request and engage the guest allowing both to converse to the same file stream. The only requirement to facilitate this is relaxing the referential constraint on the participant_queue conversation_id table so that zero and negative values are allowed. There will be then no need to modify the dbconversation add,retrieve or update methods for the guest as I can provided the new values to define the uniqueness of the guest pm rows.

DBConversation , add a new method that retrieves all guest pm rows that are hanging (determined by negative or zero value conversation id along with user id job id at zero meaning unengaged request) …will filter by site id.

CreateTable vendor classes updated createUser() to include now DASHBOARD_AGENT field MS, MSS, Ocl

Manually add new field to installed test db’s (mysql2 most important)



User class, new field Boolean int “dashboard_agent” , new get/sets and toXml() variants

DBUser new field and method changes in CRUD methods

Site should have field for allowing or disabling guest pm dashboard, when disabled guest pm links on client sites will link to a “the site is currently not accepting guest pm’s try again at a later date”, this will allow turning on and off the customer service handling based on desired metrics. For authenticated dashboard agents the enav will not display the link to the dashboard since no requests would be pending.

Enav.jsp update to include generation of new link for dashboard when user is agent and site allows use of dashboard.

Site class, new field Boolean int “use_pm_dashboard”, new get/sets and toXml() variants

DBSite new field and method changes in CRUD methods

_user.jsp new form element for processing dashboard_agent field.

_site.jsp new form element for processing
Enable_dashboard field.

DPSConfig class update the createUser() method to enable pm_dashboard_agent for superuser by default


Test new field through UI

Converse.jsp update to allow creating of guest pm will need to specify unique participant_queue row ..possibly involving both the remote ip address…must ensure new guest pm requests terminate old ones (to prevent dos like attacks from nefarious live agents)…since converse already has live verification logic this threat is significantly reduced compared to competition.

Login.jsp template sometimes returns(when the db is acting up) a session with a zero job id, flag this for redirection to kill the session.

Converse.jsp should have attribute for guestsite name , so that the url itself will indicate the site that the guest window is allowing attachment too….so the url will have the siteid and the sitename , the site id being used of course to associate the pm request row with the correct site…the guest name that the guest uses in the room will be set to the guest user object created and used in the room

On the dashboard side, a link in the dashboard jsp will allow generation of guest pm window code …formatted as described previously (with siteid and sitename) (the url should have the same pop up format as the url for remote access to a conference room generated in the emit.jsp template …see that template to use as reference)

New dashboard.jsp template , should contain page very much like imail.jsp accept listing will be a live query of “hanging” guest pm requests I can use a dynamically updated window to reexecute the query ever few seconds to keep it fresh. Users who are dashboard agents with all global site or conversation permissions will be able to select the site id dashboard to view…users without this implicit or explicit permission will only be able to manage incoming requests to their site id.

Make sure to add custom task to indicate dashboard engage actions by user agents…these metrics will be important for clients to use to gauge the productivity of their agents handling of incoming requests. THIS REQUIRES that the code for the guest request also indicates the action task to an audit trail, the best way would be to keep a log for the dashboard actions (requests and engages )

Site, DBSite, createSite() update to include new field IM_ROOM_ADS , which indicates the default option for display of room advertisements for private messages on a per site basis…this allows site administrators to determine if ad serving is enabled (assuming they have the permission) I realized this was missing when setting the default pm objects for guest pm requests…should take an hour to add in and test, then add as default value for code that instantiates pm rooms.

Change header of dashboard.jsp template so that date is replaced by “agent” ….and a green “engage guest!” displays in the status column when a request is hanging, when it is in progress, the agent column will display an active link of the dashboard agent handling the request…just like pm’s a single agent will be able to handle multiple guest pm requests this is going to be soooo God!

“drop” link would allow killing unattached requests

need a way to drop unanswered hanging requests abandoned by the remote guest (the session should clean it up on unbounding)..make sure this works.

Emit.jsp new clauses to send guest pm requests and answer via guest pm responses.

Request will open a window with a hanging communication stream via a row in the participant_queue

Response will close a targeted unique open stream (ip/type) and pop a window for responder to communicate with guest


So, there it is, currently I am in the process of checking off the items for completion. I usually do this by simply adding in bold "complete!" or "done!" after each item. Pretty low tech, no Microsoft project files, milestones and timelines...just a list of items to do! Works for me, I've added a feature to my collaboration API that forms the foundation of entire businesses (LivePerson is a leading supplier in this space) in 7 days, not bad at all! ;)

a billion is big!

I just finished quashing a bug that gave me the run down for the last two hours, as is usually the case it derived from recently changed code. The cause of the bug was my omission of a line of code that initializes an important value to a non zero integer. I added the line and voila it worked great. Along the way I was looking at the database table column values and thinking about the number of unique values that a chosen database (testing on MySql currently) can provide an integer before it goes screwy (this is a major flaw of databases that I wish the vendors would get around solving, it should not be something for the db user to ever worry about..but that is fodder for another post). I don't know the actual max value for a standard MySql "integer" off hand but I am sure it is larger than a billion. (Just googled it, it is a range of about 5 billion centered around 0) Just for no reason I wanted to get an idea of just how big a billion is(I've done it before but I am bored and maybe you haven't thought about..*grin* ), so I wondered , how long would it take for me to count to a billion if I counted one integer per second?

Answer courtesy of the windows calculator:

1,000,000,000 seconds

= 1,000,000,000 seconds / 60 seconds = 16666666.66666666 minutes

= 16666666.66666 minutes / 60 minutes = 277777.777777 hours

= 277777.777777 hours / 24 hours = 11,574.07407 days

= 11574 days / 364.25 days = 31.77508 years!!

..and that is just a billion. So a billion seconds ago it was 1976, amazing. Like I said, a billion is big, now think of all those bits on your 30 gig (billion) music player! It's a wonder you don't grow old waiting to play the music that is on it. ;)

28 February, 2008

Complexity in the Universe, conservation of energy and object orientation...

The following article was originally a post written at RD.net but I thought it should be copied here for non members to read. Enjoy!

Original link here

I seem to have added flames to a controversy with one of the posters in that thread , rainbow. I just realized that Steve Zara and others took up the standard and continued to present information to support the presented ideas. (I got really busy after posting the original and neglected to come back to respond directly to rainbow.)

Out of curiosity I just went over rainbows' other contributions to the site, turns out that the entire set of posts are to that one thread. It seems rainbow is a pseudonym for a hit and run poster, some one who created an account simply to comment on the post topic. Now there is nothing about this, often I've started new accounts to provide an alternate view but since that thread rainbow has been MIA from the site, every post is restricted to that thread. This gives me the idea combined with the points made that rainbow was a shill. He demonstrated a wide knowledge of some of the issues but lacked the critical reasoning to refrain from drawing certain conclusions (for example he assumed that the early biomolecules had no method of motility and this is patently false) based on false assumptions. Interesting, no matter my post in full follows below, you can click on the link provided above to read both rainbow's posts before it and the responses of others to his objections after for a good bit of entertainment. ;)



Regarding this discussion on the origins of life. I think rainbow is assuming that the chemistry of the early Earth was as inhomogeneous as it is today. This is a faulty assumption according to the fossil record, the planet was amazingly homogeneous in chemistry for a long long time. We know that life in the form of stromatolites were thriving along shores all along the forming continents (they were still being accreated from the interaction of the volcanic eruptions of the Earth's crust and the early seas) as early as 3.2 billion years ago. That is an astonishingly long time, but as wild as that is the very Earth itself had only formed barely a billion and a half year earlier.



I think the more likely process that led to abiogenisis was that the much more homogeneous chemistry allowed for a massive experimental space in which the early molecules could combine and recombine to form the first early protein chains and replicators. Once the first cannibalizing replicators emerged they would have rapidly spread through out the then much more homogeneous environment consuming useful submolecules. You have to look at the early Earth as almost a single global ecosystem. Today the Earth's biodiversity owes a significant reason for its existence to the great variety in biomes under which natural selection processes can proceed. In the early Earth the diversity was not in phenotypes selected through natural selection but rather in molecular types being nothing more than standard chemical affinities. Life today is highly segregated compared to the progenitors of life which being no different from complex chemical molecules could combine and recombine freely due to the laws of chemistry. From this perspective there was a much higher likelihood of chance molecular interactions producing new more complicated molecules. If we see these early molecules as "life" the natural selection was induced by the laws of chemistry providing a gigantic potential variability in the results compared to the present day biodiversity which may "look" far more diverse but in reality is much less so (if you think about it we all, from bacteria to bull are nothing more than genetic iterations of one another, highly conservative in energy due to our common use of structures that emerged in the billions of years that preceded the first visible signs of biodiversity during the Cambrian 542 mya.



If you look at a cell you see that the internal systems are more or less autonomous, they only require a particular environment and raw materials under which to carry out their chemical actions. Golgi bodies, mitochondria, ribosomes these simple engines perform specific tasks and do so without aim or direction so long as they have the necessary raw materials. The usefulness of their actions only makes sense in the context of the internals of the cell. Ironically, ID proponents claim that subcomponents have no use other then for which they were designed. I believe that for some relationships such as the relationship between a cell and its mitochondria for example, evolution itself predicts what the ID proponents conclude but for other reasons. Mitochondria are internally autonomous because the cytoplasm in which they thrive is the only remnant of the early environments in which their progenitors formed. Those environments only exist today in cells so we should not be surprised that we find them no where else. Note though that we still see that organisms amazingly similar to mitochondria exist outside cells as bacteria replete with similar DNA, the similarity is all we need to affirm the hypothesis of evolution. Just as we see symbiotic relationships between more advanced organisms today, it is very likely that the cell arose from just such beneficial symbiotic relationships arising between the various types of machinery that arose in the early biomes.



Different replicators in the form of the progenitors of the organelles coming together in order to enhance survivability by pooling resources or being forced together after being consumed by a parent organelle(the ancestors of the cell membrane or wall).



Tangentially in object oriented programming we perform this task as a matter of conservation of code, composition of class instances inside parent class objects allows us to use the attributes of the consumed objects without paying the penalty of having to recode the associated methods. When I look at a cell I see a superclass with composed child objects of other class types, by composing the objects the parent object avoids the energy expenditure of designing the consumed objects sub mechanisms. It simply need provide the environment and raw materials or in the programming case (data) required of the sub objects in order to have the sub object "catalyze" or process the data to yield a desired output data or perform some function. A good OO programmer composes well designed subobjects into parent objects to reduce the total code that must be written. The smaller the code, the faster the desired object performs its functions, the more such objects can be loaded into available memory and the faster the over all application runs. In this example the code maps to the data in the DNA but the possibilities for created functionality go up exponentially when the composed objects themselves have complex code behind their formation. In other words a cell would have taken possibly tens of billions of years to engineer a rhibosome through natural selection as the rhibosome is a relatively complex object but in the early soup the chemistry must have made such molecules exceedingly likely to form even if we have yet to form them. We can't assume that because we see a small set of internal organelles in living things today that this is all their was. As time went forward and the molecules became more complex energy would be conserved by interaction. Just as conservation of energy leads a ball down a slope using a particular path depending on the curvature of that slope and the energy in the ball, so to did the early molecules seek to conserve energy by interacting. Computer scientists can and have created simulations of the emergence of complex relationships such as these using interacting programs called cellular automatons. Though the relationships that arise are far simpler than that say between a ribosome and a cell they are emergent and non deterministic. The computer scientist has no idea how the initial conditions will seed for the emergence of interesting "behavior" in the automatons.



Now someone looking for design would claim the early Earth chemistry got "help" from a designer but there is no reason to make this assumption, the chemistry, the homogeneity of the early forming conditions and the law of conservation of energy and time are all that are required. Just as a computer scientists looking at an emergent simulated environment of interacting cellular automatons does not invoke God for the complexity, there is no reason to suppose God for the biological analog simply because we have yet to replicate the conditions that gave rise to our life.



For those unfamiliar with OO, here are some links on object oriented programming:



http://en.wikipedia.org/wiki/Object_orientation



http://java.sun.com/docs/books/tutorial/java/concepts/

Complexity is brilliantly explained in the book by Nobel Laureate Murray Gell-Mann "The Quark and the Jaguar" I highly recommend it!

Get the book here

mastering multiple mountains...

I was thinking in the shower today, ( a common practice) and I began to recall an experience I have had in the past when having a discussion or debate with individuals who express views vastly different from my own in areas that are considered controversial. This particular topic was about the king of controversial topics , religion. In the past I've gotten into quite heated exchanges when discussing this topic, I have a very low tolerance for individuals that think improperly. Let me explain what I mean by improper thinking, it is the tendency to draw conclusions that do not directly derive from a given line of reasoning or evidence, people do this constantly and I would be lying if I said I haven't performed it myself now and again without knowing it. The point is, when it is pointed out that ones logic is broken, individuals in the heat of discussion would rather belabor the point in the effort of appearing to be right ...rather than concede the point and analyze their methods of conclusion to rectify the issue. In my life, I've found that being able to admit when I was wrong was the most expeditious way for me to get on the right path. What matters most to me is being aligned with what is true, not appearing to be right. Ultimately, truth is what persists, the perceptions of what is right soon fall by the way side as the unstoppable persistence of truth wears away all objections to its existence. Take the case of the round Earth, through out history the knowledge of the spherical nature of the Earth has come , and gone and come again , to different regions of the world, to different cultures. Today there are scant groups of people in the world that would deny that the Earth is a large , roughly spherical ball of water, rock and air but even this truth may still find variety of views. Take for example the inhabitants of the Andaman Islands in the Indian Ocean. The Andaman Islanders are a group of people said to derive from an ancient stock of humanity (modern humans) that left Africa around the time that homo sapiens sapiens began his trek into Europe. The Andaman Islanders share genetics and morphology of several small groups in many Asian countries. They are similar to bands in South East Asia (Laos, Cambodia, Burma, Malaysia), the Philippines and form the stock from which modern New Guinea and Australian Aboriginal populations are derived from. Here is a photograph of an Andaman Islander:





The interesting thing about these "negrito" people is that , to this day, many of them have strongly rejected the contact attempts of other humans. From the days of European colonial expansion into the region, to the modern governance of the Islands as a possession of India, the people on some of these Islands have rarely accepted visitation from modern people. This being the case, some of the islands have not been subjected to the conquer and conversion methods employed by colonial forces and thus maintain the practices and beliefs of their past to this day. Surely to an Andaman Islander, the modern individual is a conundrum, a mystical people with weird skin and eyes and weirder food and clothing practices. We don't question the fact that these people will likely have no conception of the fact that the planet we live on is an orb spinning about a nuclear furnace in the sun. Their understanding of the world is limited by their vantage point, they have one mountain top from which they are seeing the world and to them, that world is all there is and all that needs to be The comfort that they have with their way of life manifests directly from their restricted knowledge and view of the world, so is it true with all areas of knowledge. Our perspective of the landscape (here used metaphorically) that defines any area of knowledge that we analyze is dictated by how much of that landscape we are able to "see", in this case experience or relate to existing facts gathered about the world.

I think this difference in perspective as necessarily created by differences in the acquired levels of knowledge that we have of different areas of knowledge is what leads to such strong arguments on subjects such as religion. The reason is that as the subject moves more and more from the ability to be tested empirically and locally, it becomes more likely to be explained in terms of the metaphysical or the non empirical "solutions" that have been good enough to explain our survival in the world up to this point. The fact that the Andamanese still exist today and still on some Islands practice and retain their native beliefs oblivious to the existence of other forms of belief, is a testament to the fact that all that is needed to survive and indeed substantiate the efficacy of a form of belief is that it promotes survival. Even if those beliefs only indirectly promote it , for example, the rituals that many cultures practice to ward away disease or inclement weather or ensure bountiful yields are associated with survival even if they don't succeed in correlating with it 100%. People have an amazingly tenuous set of reasons to accept something as effective, one event of correspondence with a belief is enough to give it reason to be held even in the face of dozens of subsequent failures.

This takes us back to the idea of the mountain of perspective, the Andamanese resist modern culture so strongly precisely because they are incapable of seeing anything else, incapable of comprehending anything else. The remote location of their islands ensures that the benefits of modern life that has spread to most large continents are still mostly absent in their islands. Similar persistence of ancient beliefs exist to lesser degrees in the remote South Pacific Melanesian Islands. If ones view of the world is sufficient to explain their survival then there is no need to supplant those beliefs with anything else, there is also no need to investigate other perspectives. This gets back to the arguments of religion, the key reason that I have such vehement arguments with such individuals lies not in the fact that we both want to be right in the argument , and to a degree we do, but it is more so tied to the fact that we both believe that our "facts" on the subject are sufficient to show we are right..ignorant of the true fact that we both have different mountains of evidence and data from which we are drawing our argument. That being said, the truth can be approximated only by mastering multiple mountains of knowledge that touch at their disparate edges. It is through this act that we lift our vantage point from the certainty of a restricted dogma based on "what has worked in the past" to a more expansive understanding of what is true. In religion the main differences in the argument I've noticed boil down to a few things:

  1. Difference in the knowledge of history between those in conversation.
  2. Differences in the use and meaning of words.
  3. Failure to refrain from generalized statements or qualify statements properly.
  4. Differences in the knowledge of areas of investigation that invalidate otherwise "obvious" beliefs that are wrong. (For example we know the Earth is round despite the fact that it looks as if it is not from most vantage points.)
  5. The habit of people to want to appear right in the argument rather than actually put forward truth.

I have found that by discussing topics dispassionately and with the 5 points presented above in mind I have been far more effective in getting those I have discussions with to consider my argument by standing on my multiple mountains of information vicariously. I do this by posing questions that can only be answered by fully understanding those other areas of knowledge. Nothing gives a speaker pause than to realize that there is so much about so many things that he or she is ignorant of. What is important is the presentation of the data in a manner that is not considered offensive (especially in a public debate) to the individual. I must admit this is vastly easier said than done! A humble presentation of data , and an insistence on the part of the listener to go research and confirm that data themselves, is all that is needed to put the seeds of interest in most people. Some times, the gulf of knowledge between the individuals is so great that this seems impossible, but patience and an appeal to acquiring knowledge from multiple areas can erode this gulf and thus allow a competent discussion to be had. By introducing the idea of multiple mountains of knowledge to individuals that formerly believed there was only one, we expand their world view and allow them to ascend the ladder of truth while escaping the bit of empirically devoid but effective in spurring survival, belief.

You can read about the Andaman Islanders here:
http://www.andaman.org/

http://en.wikipedia.org/wiki/Andaman_Islands

27 February, 2008

considerations during design

The software design process boils down to a few key abilities:

  • Defining the problem scope.
  • Targeting the most applicable solution domain for the scope of the problem. Scope encompasses all extremes of the problem space, from the unlikely and rare scenarios to the very common scenarios, these extremes moderate the demands for resources, disk, processor and bandwidth. The art of good design lies in knowing how to tailor your solutions to the most applicable solution domain for the problem at hand.
  • Implementing the solution for the applicable domain of importance that is most efficient as opposed to most expedient.
  • If multiple solution domains must be covered ensuring seamless transition of solutions from one algorithm to the next.

The first and second points are most important, as you won't know how best to solve a problem if you can't define its' extents. Many times developers are unable to put their fingers on all aspects to a problem, this is unfortunate as it may severely restrict the solution they engineer as their ignorance of aspects of the problem that can be exploited for symmetry leads them to make inferior algorithm choices for the solution domain. How do you determine a problems extents? You make sure you test it at the extremes of performance using an old tool, a thought experiment. Consider a very unlikely high load or activity situation and then roughly define an algorithm that solves it (one solution domain), then consider the opposite , very low load and determine the optimum solution domain for that problem regime, finally determine a middle of the road situation and define a solution for that..depending on the problem, you may find that a given solution is optimal across the entire problem space or it may indeed require separate optimizations within the problem scope. Once you are done testing these edges or the event horizon of the problem, you have covered all the realizable conditions and thus can be assured you are engineering optimum solutions even if the eventual algorithm for most cases will not extend into the extreme scenarios discovered. In fact the act of defining the problem already sets you on the road to the solution, as by this task you also determine which of the identified solution domains are the most likely use case for the load, resource and bandwidth constraints of the final implementation.

The next two points cover the implementation which I like to call popcorn, the solution domains have been isolated, the optimal domain(s) for the problem in question explored using the previous intellectual muscle work and now it is time to just do the grunt work of building it. Now the mind shifts from looking at big picture concerns of latency between servers in a cluster to little picture concerns local to the executing machine during run time. A good example is in noticing how the choice of a variable declaration as static can effect memory utilization on a machine, other similar concerns are the choice of implementing a method as a concrete or as a forced overridden method from a abstract base class or interface inheritance. These choices can hinder or help the execution of code efficiently on any given system. One that I tend to pay particular attention to is byte size, every character in your code that is not needed is memory taken during execution, under loaded conditions these bytes add up to significant performance reduction so making your code as tight as possible through extreme parsimony of characters directly benefits efficiency in the long run. The rule of thumb I use is , use as many characters as required to ensure intelligibility of the code and no more. Another major source of issues lies in making classes too big, a class should only contain methods that are inately associated with the class. It makes sense for a "File" class to have a "read" method but it probably doesn't make sense for it to have its own "copy to" (as "copy to" should be something you do to Files not that Files do to themselves, a very subtle distinction), also note when a function that you wish to add to a class could also be useful to other classes. These generalized functions are better off in a static Utilities class where they can be employed on the different classes that need them, and where the static nature of the class ensures minimal loading of the method code for each instance of classes that employ the function. For example, if "copy to" was implemented in "File" it would be loaded to memory every time a File was instanced, taking up critical resources to do it, under load this seemingly small difference could prematurely curtail performance and directly impact operating cost. Moreover, loading the class instance (and all its methods) does not guarantee they will be used...so the loading is waste for most cases (especially for a File class where you most likely want to read from it, write to it rather than copy it to some location) By having the "copy to" method in a static Utility you ensure that it is highly likely to be used over the landscape and lifecycle of ALL classes that are in your class hierarchy that may require the function.

Finally, and related to the last point of putting methods where they are most likely to be used, is the idea of just in time coding, you want to make sure that when you load something you will be using ALL of its code, if you don't then you should consider loading the thing in parts (or exporting the functions that aren't used to static classes as mentioned previously) what this ends up doing is it makes your class hierarchy wide but not tall. Meaning your memory profile looks like loading of many LITTLE classes rather than loading of few LARGE classes. Under loaded conditions the latter option is far more inefficient with resources than the former so remember , many Little is better than few Large when it comes to classes (or any atomically executed code unit; html page, javascript, jsp templates...etc.) Considerations like these are the ones that refine the big picture solution domains into solutions that highly conform to the over all problem over its entire scope and yield an optimal and elegant solution. As with anything else , practice performing the steps on real problems gains one facility at performing the tasks.

Deep vents and shrimp that refuse to fry!

This link to an article on a newly discovered hot vent (as of late 2006) is interesting. The deep sea shrimp species discovered live in amazing extremes of temperature. Adaptive evolution on amazing display.

Take a look!

Programming biology

The following is an excerpt from an email I sent to some friends on 24 March of last year. It concerns an advance in biology (one of many ) that occurred last year.

http://www.iaucsb.edu/pa/display.aspx?pkey=1571

The following text was my comment regarding the link:

As if they simply took the "class" for the extra color receptor and used it as the "base class" for receptor development in the modified mice eyes. Just as we take a superclass and by inheritance endow subclasses to use it's attributes in java or enable multiple inheritance in C++. I saw these similarities first when I was in high school between electronic systems design and biology and then again saw it between OO programming and biology and electronics systems design. It is possible the original OO language designers may have had the biological mechanisms in mind when they came up with their ideas as much of this was still being teased out during the late 70's and 80's when OO hit it's stride and the terms used have uncanny similarity. "Inheritance", "polymorphism", "interfaces", "encapsulation" all have corresponding meanings in biological systems. Quite cool how man has come , through his design of programming languages to similar principles of reuse of code, (which ultimately enables conservation of energy in the end results of both endeavors ...surprise, surprise) as mirrored so closely in biological systems.

Since I started programming in OO in the early 90's I've always found the biological analogs (knowingly or not by the inventors) used in it's construction intriguing. As geneticists continue to figure out which genes encode for which biological system or function , genetics will take on the industrial efficiency of software engineering as organisms will be built up or modified by genetics engineers working in genetics development environments very similar to the IDE's we use in OO coding. Imagine writing the code for an organism and "build" -ing it (in a vat of goo) just as we write code for applications today (well minus the vat of goo)...the day is fast approaching, will we be able to handle it or will the power at our finger tips destroy us? At least in the realm of software engineering our creations (the software) have only limited ability to cause havoc (with in computer systems) but what of a built from scratch super organism, say a roach with a titanium reinforced exoskeleton that can't be crushed ??? (I don't know why someone would make it but if it's possible ...why not? people have written lots of equally useless software programs that is for sure.) A couple of years ago I read an article by Sun development guru Bill Joy, in it he expounded on his fears for humanity being able to control the power of genetic modification that we are currently unleashing. I tend to be a bit more optimistic and think that hopefully we'll be mature enough to deal with the ramifications of our playing in god's sand box as first class creators of life but I am with Joy in thinking our chances of coming out on top are less than they were 10 years ago before we mapped the human and many other species genomes. I try to allay my fears by citing the fact that it's been over 70 years that we've been living with nuclear bombs and only once (one conflict , two cities ;)) have they been used offensively. However we now live in a world were radical religious elements have openly expressed a desire to strike at their enemies using any and all acquirable means. At the same time we charge forward in labs all over the world to unravel the mysteries of the human proteome in order to really make programmatic construction of living creatures possible, a nuke as powerful as we've made them, only tend to destroy and affect a limited area. Sure , fallout and residual radiation kill long after the blast but eventually they go away. A biologically engineered super organism could succeed in wiping us out ...us all out (human species as well as other species) if the power to create them so easily falls into the wrong hands. Nukes are very difficult to design and build or acquire and will stay that way, the tools and materials and expertise necessary for genetic engineering are orders of magnitude cheaper and accessible and far more powerful if used to nefarious purpose. Will we be able to survive the genetic engineering cold war to come?

Regards,

Dsl

Life of a Cell video and other biology resources...

Harvard has a set of excellent online resources for detailed technical information concerning countless topics in science. Last year I came across the multimedia site for the biology department through a video called Inner Life this video shows in visual CGI glory the beautiful dance of biomolecules inside the cell. I have a very visual memory and the processes that I read about and could only imagine when I was in HS are put to beautiful motion in this video. Check out the different versions on the page with and without narration. The myocin molecule walking animation is an amazing demonstrating of a cyclic biomechanical operation that occurs simply because of a periodic affinity and bond energy change between the operative molecules...so beautiful. My favorite part was the mRna to protein synthesis segment of the animation, just like I always imagined it! The page also has animations of other cellular functions as well as demonstrations of conservation of energy principles. (which ultimately are the root source of ALL time evolved change weather it be in living or non living things)

Another resource I came across in 2005 is a general primer on DNA, mRNA transcription, and protein synthesis. Animations make the concepts described easy to comprehend.

http://www.ncc.gmu.edu/dna/index.htm

Enjoy!

take a look and enjoy!

the sent2null space header image challenge

The new header combines several of my most cherished endeavors. The background image is a Hubble Space Telescope image of a portion (n44c) of the Large Magellanic Cloud , one of our companion galaxy clusters. Streamers of illuminated gas flowing in the stellar winds of a bright star. I have spent a lot of time at the Hubble site downloading the massive full resolution images for printing at large size for display on various walls in my home. I spread the gospel about the site (each image is completely documented) to everyone that cares (or doesn't care) to listen. Head over to it if you haven't and get to downloading...I have a 20" x 30" print of the galaxy M104 "the sombrero" that quite literally brought tears to my eyes. It is important that we experience these images to simultaneously realize the grandness of the universe and our small part in its grand symphony. The title text , "sent2null space" of the image was rendered using a custom designed font (a fun little hobby for those that always wanted to make their own font families and use them on their pc) I use a combination of tools to create a new font.

  • Graph paper (printed on my printer from free pdf templates that you can custom make to your requirements) Get it now.
  • Once the font is rendered to the graph paper, I use my trusty 10 year old microtek scanner to scan the character sheet into the pc. Rhino a NURBS based 2D/3D modeling tool allows tracing the curves precisely and export .ai files (Adobe Illustrator)
  • Adobe Photoshop the workhorse of graphics professionals (that I've been using since version 3) to scale, flip, copy , color the .ai files to black clean edged characters.
  • Font Creator is used to generate the font set (true type font) and import the set into windows (or any other platform that supports true type fonts)


The final result is a font, below a couple of the finished tiles ready to be imported into font creator to render the font:

















Finally, the red symbols in the lower right corner represent something that will be familiar to fans. I wonder if any one knows??? ;)

Favorite books post...

To those that got here after reading my post at Richard Dawkins.net I wanted to start a discussion about books , fiction and non fiction and what they mean to you. In this increasingly digitally delivered world, I find reading a good book a great way to clear the mind in order to fill it with other ideas. The books I've read have shaped a great deal of my world view, unlike many people I know I don't read as voraciously (well not non technical subjects anyway) I wonder how close you are to your books? Are you a reader and giver or a reader and keeper? Books I love , I can't part with, I still own the texts that I used in High School (which I was able to procure after the fact) I also have boxes filled with the many (expensive!) texts that I had to buy as I acquired my BS EE degree. How do you treat your books?

26 February, 2008

bad design everywhere!

I was not as productive today on my continued guest private message implementation as I wanted to be, I was just getting into the groove, lining up my wheels to the highway of efficient code when out of the blue came....

my nephew.

See, two days ago he mentioned that his pc was on the fritz, since his loving Uncle is the "pc guy", he came over to ask me if i could take a look at it. At the time I was just getting the implementation started and was very busy, I told him after a short moment of thought "Tuesday." Of course when he did show up, I was completely surprised to see him...long story short, I spent the next 5 hours reinstalling XP. This takes me to the subject of this post.

Microsoft (and many companies) are plagued by a problem of design of large technical products that has to do with the scale of the task that is exacerbated as the number of individual agents working on the tasks sub parts goes up. Take an operating system, Microsoft has several teams working simultaneously to design their operating systems. Outwardly it might seem that having multiple teams working at the same time will drastically reduce the time needed to complete the entire task BUT it comes at the cost of necessarily reduced quality. The quality reduction comes from the fact that at each interface between the teams (and within teams ..interface (or interaction) between developers on the team) there is a loss of information or a difficulty to transfer information critical to ensuring that the finished whole works efficiently. The use of object oriented programming methods in theory is supposed to stream line the component based design that is critical to making software work efficiently and with as little code as possible BUT when you have that design process interrupted by different teams , with different class design strategies (using sometimes different tools), the big picture tends to get cluttered away with gunk in code. This gunk is what leads to the miscellaneous kludges that must be added to fix the slight (or major) disjoints that occur when the many different components of a large design are sewed together, like some impossibly large and complex patchwork quilt.

A particular item of observed bad design that piqued my annoyance concerned what I considered easily fixed UI problems. The XP OS dialog for turning on and off the firewall and windows update had a nice graphical illuminated button , all ready to be pressed. Of course pressing the button does nothing...instead , you must scroll the panel down to links to disable or enable the features. I wondered why wouldn't they just allow the buttons to be depressed? The user is expecting that intuitively due to the graphical paradigm employed yet instead a more complicated process was created to perform the task. The next item was more dangerous, it involved the messenger service of windows XP, which as most people familiar with windows XP knows is turned on my default. The amazing thing was that within seconds of installing the computer to the broadband network I received two different messages that claimed that my registry was corrupted and that I should go to a specified web site (not microsoft of course) to download the "fix". We all know what that was, a network delivered piece of malware that identified and sent a messenger service message to the pc within seconds of it being online. I can't imagine how many pc owners are upgrading XP at this moment, are getting that message from their local network and then are studiously clicking "ok" to download the rootkit/malware onto their computer. All this happening simply because of Microsoft's bone headed decision to install windows XP with the messenger service (as well as other exploitable but rarely used services) turned on by default.

The bright side of bad design though is that when it is found, it should be taken as an opportunity to solve the problem with the right solution or go build a company around doing the same. If we hone our ability to detect the whiffs of bad design that pollute the world of goods and services in which we are embedded and simultaneously engineer good design we can put ourself in a place to profit from it.

25 February, 2008

another unforseen stop...

As indicated by the subject line, after the last post I went ahead and tested a spur of the moment addition from this morning. The conference room UI lacked a simple switch for enabling or disabling the use of shared files on the table for the Room. I decided this should be added to provide administrators the novel ability of selectively providing access to table files in a conference room by sending out requests at different times. For example, a User can now set their room to enable table files, invite 5 people and then disable table files. So long as the 5 other participants join the conference before the file table is disabled they will receive sessions with the permission enabled. All subsequently entering participants will thus be unable to view, download or upload files to the table...this will allow designated groups of individuals the ability to share files while still conversing and collaborating with other participants who for whatever reason should not have access to the files on the table. I anticipate use of the room in this way will allow a close tie in of collaboration processes performed in the real world to those in the digital world through the collaboration API. The use of multiple independent attributes allows an exponential number of possible collaboration states between Users/Guests and the Files shared while communicating in real time. Tomorrow the bulk of the session tear up/down code goes into place!

a detour into offline messages...

A key feature that I recently added to the collaboration API (about 10 days ago) is an offline messaging capability. If you are a user of yahoo messenger you know how this works, when you are offline users are able to still open an IM window and send a message that you can then read when you log back into the system. I wanted to perform something like this since I finished the main implementation of the collaboration API almost a year and a half ago but I pushed it back , now that I am near the end I realized the 3 days it would take to code it was well worth the potential benefit it would bring to the users (and indirectly to the company!)

The offline messaging allows each User to control who can send them offline messages, unlike the yahoo thick client in which I periodically receive spam messages from users who are not on my contacts list. In my implementation, a User has 4 options that other Users can be given with regard to offline messages:

  • Offline messages disabled
  • Contacts offline messages only, notification off
  • Contacts offline messages only, notification on
  • Any User offline messages, notification off
  • Any User offline messages, notification on

In addition to these options a User has a stealth mode option for when they are online, this allows them to come online silently without notifying their contacts (or any Users browsing profiles) of their presence. If a User is in stealth mode, naturally they appear offline to the offline messaging code and thus are subject to receiving the offline messages while online. This gives them the discretion to do nothing, respond to the User by private message or email. The User Settings page displays the status of Users based on the above mentioned settings, I discovered that the logic was all screwy...the offline notification settings were mixed up and would indicate a User was online when he should have been indicated as offline, when online the display link allows sending a private message as opposed to an offline message even if offline messages are disabled (this was another pathology). After looking into the code I realized one thing quickly, I need to clean up a bit! The conditional statements were not aligned very well, I soon discovered the issue and fixed it. I noticed one aspect of symmetry observant designs that employ method polymorphism and combinations of hierarchical and compositional class relationships is that the savings on the class level can turn into some interesting conditional relationships. The use of simple integers as attribute fields in the classes saves a great deal of code in that the class ends up having many attributes but these various dimensions when put together in the client code inspire the creation of conditional statements to build the desired behavior. Case in point with regard to the problem mentioned above was that several attributes of the user object were conditionally tested to determine which of the links (offline message or private message or conference or eject...etc.) displayed on the settings page, this led to some clauses containing up to 3 sub conditions, though this results in rather imposing looking conditions at first...with proper condition design (making sure to take advantage of short circuiting) this leads to rapidly executing conditions tailored to exit at the rate of highest likelihood given the condition sub conditions.

All that said, I am still noticing a great deal of places for optimization of the code to squeeze it down to as few essential bits (characters) as possible. Since the client code in my framework runs mostly in jsp templates, squeezing down the bits directly aids performance over the distributed cluster in the long run and that directly reduces costs for operating the business...all great things.
Time to get back to the implementation...

still crunching....

I got to work today as soon as I got back from my morning run and was ready to go after a nice power breakfast. As mentioned in the last post I spent yesterday working on the dashboard template. It required addition of a new method for retrieving the extant guest private message requests on the participant_queue table. This new method was made with three attributes, the new site_id column , the conversation_id (used as the guest pm uniqueness specifier) and a boolean attribute that determines if engaged (meaning a User is conversing with the guest) or unengaged rows are returned. The returned rows are extracted as an Object[] of String[] objects. In other classes of the API I've found it useful to return specific sets of data from the db in an object array when there is no formal need for a managing class. In this case, we only use the participant_queue rows to display in the dashboard, all other use of rows (public/private, user/guest conversations) is implicitly performed by the other API code. Thus the time benefit is optimal (since it is fast) and the memory hit is optimal since we don't have additional object instance and destroy actions to incur processing and memory load during the dashboard view action. I partially finished the implementation and called it a night ...

Today, I started by analyzing the problem of concurrent guest requests across multiple sites, since the framework is a multi tenant frame work it is possible for multiple sites to provide guest PM capability independently. The addition of site_id to the participant_queue table thus gives us support of multi tenancy for free. (This happens often with symmetry observant designs) I had to go into the conversation db access class and update the add,update and remove methods to support the new site_id field by simply expanding the attribute lists of the existing methods in new overloaded methods. This will allow multiple sites to enable guest private message requests and allow a single guest (IP) the ability to make requests to multiple sites hosted on the framework. In multi tenant design this is not only important it is critical, for it enables the possibility of a user making a guest request to one site that might be licensing the use of guest functionality while simultaneously making a request to another site and converse with both sites unaware of the fact that the guest private message facility they are using comes from one provider. ;)

I compiled and tested the changes made , many - small as opposed to few - large, and was able to engage normal User or Guest conference sessions without issue. Next stop is adding in the actual guest session tear up an tear down code which will require a little bit more brain work...

23 February, 2008

refinement to the solution...

burning along with my implementation of guest private message addition to the collaboration API...discovered that contrary to the assertion made in the last post that I wouldn't need to update any of the conversation db access methods , I realized that for purposes of display in the dashboard.jsp template that will allow guest private message agents to manage incoming requests, I will need a method that performs the relevant db query. I also noticed that the participant_queue table which tracks the active conversation participant data did not have a column for site_id. Turns out having the site_id is critical for providing filtering by site_id and thus enabling an efficient multi-tenant management capability for this new feature..so I had to add the new column in the classes for the supported db vendors. I added the missing bits and then went on to start working on the dashboard.jsp template before getting back to the actual logic for setting up and tearing down the guest private message sessions for the requesting guest and the designated agent. So though the symmetry reduced the work considerably from what would have been required otherwise, it still required a little bit of tweaking. Ah popcorn!

22 February, 2008

Interlude: the iphone browser

I was able to test the AJAX driven interface of my collaboration API on an IPhone's safari browser today. It was my first time using the IPhone and the first time I used authenticated to my test site using a handheld device. Though the UI of the Iphone is a bit hampering in how it provides navigation from form fields and how it engages submittal it was still successful in enabling all of the functions of the site. I was a bit surprised that it worked perfectly , I was expecting at least one rendering glitch or issue having to do with the execution of the javascript but everything turned out perfectly. Though adherence to web standards on my side of the web was employed to minimize issues, I'll have to give a good part of the thanks to the Safari web browser on the IPhone which does a good job of supporting web standards.

Kudos to the IPhone.

crunching pop corn...

I am in the midst of the implementation of the solution mentioned at the end of sent2null space: objects , abstractions, symmetry and pop corn??? I ran up against an issue that happens occasionally when a modification is made to a class model. You'll recall , I realized that the uniqueness of guest invitations could be used to define the conference type and allow an existing conference to be used as proxy...while investigating deeper, I realized there is no need even for the proxy if I change the structural rules behind the table that tracks conversation participation. In my framework this table is called appropriately the participant_queue it simply lists the attributes of Users engaged in conversations. The database schema for the table enforces referential integrity for a column called conversation_id , in the original implementation of the collaboration API the idea of guest Private messages was not supported, only guest conference users were supported and since a guest to a conference is always joined to an existing conference on the system , referential integrity for conference id's made sense ...but to support guest PM's I could either add a new type and then create new managed conferences for every guest PM request as mentioned in the previous post but this is computationally as well as resource inefficient, especially as large numbers of guest PM requests come in. To allow for efficient guest PM's an unmanaged interaction would be ideal, the solution as mentioned previously was to use the uniqueness of guest pm requests...I realize now that the minimal effort and maximal pay off comes from relaxing the referential integrity constraint of the table column conversation_id and using the existing methods for adding , retrieving and updating conversation rows based on the participant attributes. This single action allows me to readily support the specific symmetry of guest PM's without requiring managed conversations (with valid conversation id's) to be used as proxy, by using zero value conversation id values to correspond to guest PM participant rows in the conversation_id column I can track guest PM's independent of valid managed conversation objects (those with positive integer id values) the drawback to this is only that I have to recompile my existing test instance of the application and reinstall it (strictly speaking I don't since I can modify the constraint via the db admin tool) anyway I decided to just make the change and forgo all the other hassles of needing to update the add, update and retrieval methods of the conversation objects. So this is a perfect example of observing a particular symmetry in action, in this case it is the symmetry of the database schema when the table constraint is in place, I couldn't use negative or zero values...but by relaxing that constraint use of zero or negative values allows me to employ them for the use of the guest PM's which now can be instanced entirely in memory without management resource allocation on the db which is a key drain in performance especially under scaled conditions.

I'll be explaining more about the actual implementation as I role through it in the next few days.

engineering in art


In order to continue an explanation of the relationships I see between art and engineering I will expand a bit what I've mentioned in previous posts. The abstractions that are employed to master use of the "objects" of each discipline are the same, the process of solving a problem involves first, isolating the objects and understanding their parameters or attributes in detail, followed by building the solution with the objects , while adhering to the parameters or interface attribute constraints. In the drawing analogy , the idea of shape covers what in programming is expressed by interface.

The final step involves making a view of the objects constructed to solve a particular problem. In software engineering the view defines the scope of the solution, how efficient it should be in a given work domain, how scalable it should be, how intensive in resource utilization it should or should not be. In drawing, the view maps to the visual idea of perspective, how does the physical object appear when observed through a hypothetical camera in a given position given its known physical extent, it's translucence, it's texture or illumination. In both cases we are painting a solution from the vast possibility of options in the problem space by discretization of the properties of the view.

As mentioned in sent2null space: objects , abstractions, symmetry and pop corn??? with practice of creating the object abstractions from the problem space comes an appreciation for using symmetry to vastly reduce the work required during the implementation stage. In art this involves taking advantage of geometrical symmetry, cylindrical, spherical , ellipsoidal characteristics are seen in the objects and transformations on these primitives allow real time computation of intermediate positions of the object from a desired view. In software engineering, the symmetry prevents itself in taking advantage of constraints of the over all design to avoid doing work that is computationally or memory inefficient.

Today I had a discussion where explaining these similarities were hard to come by, though I was some what successful in my endeavor mostly the correlation between solving and art problem and solving a software object problem was hard for the listener to absorb. The difficulty in seeing the connections aside, I think a great deal of effort can be avoided by taking advantage of these ideas in any area of study. Another example should help, this one comes from my training as an electrical engineer. In electrical engineering, the task of designing an electronic or electric device involves the exact same steps of object learning, object building, object symmetry finding and finally solution synthesis.

In the ee design world, the objects are discrete components which not surprisingly have constraints on their behavior when placed in an electrical circuit or circuits. Inductors store magnetic energy as a result of the thickness and length, density and type of windings used, capacitors store electric charge based on the area of plates, their separation and again the type of material. The key to understanding how to build circuits with these components is to first understand intimately the components, what they do when fed a signal and why...this is the same as knowing how the component is "viewed" , to use the drawing analogy again, or of knowing how an object with a given attribute set and return type will behave when fed a given set of values.

Again, the act of solution synthesis involves the same abstractions, as well an eye for symmetry allows simplifications to be revealed ... standard electronic circuit design involving non linear components like capacitors and inductors , diodes and transistors were solved using large sets of simultaneous equations that expressed the instantaneous (time dependent) relationships between the circuit components. Later, methods were developed to take advantage of the continuous time response of the components , these allowed integro differential equations to be solved for specified initial conditions of time and other parameters like frequency, voltage or current and thus expedited solution.

Recognizing the symmetry of frequency in various circuits allowed the application of transformational solutions. The Fourier and Laplace transforms being key methods to reduce the order of complexity for complex electronic circuits and thus make certain hard problems easy and certain virtually impossible problems only hard to solve. Each solution method employed a deep understanding of the object relationships of the component level in conjunction with a recognition of symmetry that allowed solution short cuts to be made.

Today , no modern electronic circuit designer would think of solving complex circuit values without use of the advanced transforms usually in a simulation package like Matlab. So , as in art and software design, electronic design proceeds efficiently from understanding objects, abstracting views, utilizing symmetries and then synthesizing solutions. As I come up with more examples that correlate (one that I think of now off hand is cooking) I'll write a post to show the relationships but hopefully this latest post cleared up the idea some what...well I hope it did anyway. ;)

keeping ideas original

In the last few years, I've noticed that ideas or conjectures that I made in the past as a naive and imaginative college student have come to have substance behind them in the empirical results of various research reports in the areas that have been published since those times. I won't go over the actual items because they really aren't relevant, what is relevant and likely is that I wasn't alone in making these mental prognostications. I think that as we speak, brilliant solutions to incredibly vexing human problems are being engineered and forgotten in the minds of precocious young thinkers and in the minds of refined thinkers as well. The key difference between the two groups lies in my view in the perception of two groups that allows one to vastly out perform the other in the skill of "innovation" is self motivation or belief.

As a young college student I literally felt that there was no problem that I alone, could not solve. This belief was reinforced by the experiences I had in studying illustration, my early forays into programming as a pre teen in the early 80's and in college , the focus I placed on studying particular subjects that I found interesting. As a youth the landscape of life is barren, there is only our boundless optimism and fascination with the breadth of open questions begging to be solved. Most of us (at least here in the United States) are free of the burdens of self support thanks to the aid of our parents. Some of us even so blessed to have had the academic performance and focus to secure this support through our own energies. Free to engage the problems , youthful engineers can plunge themselves headlong into the difficulties and through dilligent exploration of the possibilities perhaps chance across novel and innovative solutions to previously open problems.

In contrast, as we move beyond the early years, we tend to saddle ourselves with the responsibilities of life. We fall in and out of love, we have children, we redirect our energies to these endeavors and necessarily are left with less ability to attack the open problems that still beckon us. This realization then fetters the mind with doubts, doubts of our ability to perform to the same level as we did in the past, doubts of our ability to keep up with the younger generation, doubts can serve as the anchor that keeps our ship from steaming forward to new explorations and discovery. The fountain of ideas that we once were is brought to a trickle and a feedback loop of self doubts in some cases , completely stops the flow. In my life I've done my best to rebel against this erosion of ideas with time by ensuring I am free of the diversions (some would say blessings but that is a matter of perspective) that would reduce my ability to solve problems with innovative and novel ideas. Rather than steep myself in family (a task I do want to engage in eventually) I steeped myself in study (of both illustration and engineering) , rather than laden myself with material burdens (home, car...etc.)

I've been quite successful living an extremely ascetic life, one that I have chosen so that I could direct my energies inward toward the problems that I deemed most important to solve. In so doing, I've realized that a second benefit to this focus is the fact that I have been in some ways oblivious to the ideas of others, now in some areas this may be a huge detriment, as being completely unaware of the progress being made in a particular area of study can lead to the world of advances passing you by, but just the right amount of sampling of the work of others , allows the ideas to catalyze rather than pollute the innovation engine in our own minds. If we focus on every detail of what others are doing I believe we tend to pick up those ideas and may then neglect novel patterns of thought that might have led us to original creations. So, to have an eye on progress periodically allows us to shuffle the deck so to speak on the choices we will make in our designs without being stuck into a rut of following the prevailing "dogma" of ideas.

In history we see that many of the most innovative achievements were performed by lone gunmen characters who seemed at first , secluded away oblivious of the workings of the world. Einstein serves as the archetypal example of such a character, but Einstein wasn't a lone gunman, he sought and maintained correspondences with many influential individuals in his work, he was able to bounce his ideas off those people and use the feedback to continue his innovations without being polluted by a mass of dogma that might have derailed his train into the unknown and extraordinary concepts of relativity, brownian motion, photo electric effect among other great realizations. Einstein kept his ideas original by NOT mingling too much in the world of scientific dogmas that said, time must be absolute, that space must have an "ether". He delved into the work and let it reveal the truths to him and objectively accepted the results. Software engineering is in many respects the same type of endeavor, it is possible to get lost in a sea of dogma, as we embrace this or that programming methodology, as we are introduced to this or that "pattern", these templates allow us to solve certain types of problems but as we all know, all problems are unique and just as a tailors suit patterns are not amenable to designing all sizes of suits...so to is it that the constantly evolving dogma of software engineering technique (one which has a definite industry behind it in the form of lectures, books and videos by the way) are not applicable to all problems. We must be facile enough with our ideas that we can engineer novel solutions on the fly without being restricted by the patterns of dogma. In doing this we ensure that we can busy ourselves with solutions that not only solve the problem, but solve it in a tailor made fashion that ensures extreme and complete long term efficiency. The theme of taking time to properly design solutions and avoiding the pollution of foreign dogma is one I've come across a great deal in my life, so far it has been very beneficial and has kept me from falling into a rut of self doubt, at least that is my perception. ;)

jist: be so busy with your own innovations that you have little time to read about the dogma of others. Time will reveal the value of the path you have taken.

objects , abstractions, symmetry and pop corn???


I started the implementation of the very last impromptu addition to the features planned for the framework. I have been mulling the day on various aspects of the implementation for the optimal and most efficient approach, as is usually the case I found myself debating issues that might seem insignificant at first blush but upon deeper reflection reveal themselves as important points.

Case in point is how I will design the guest pm dashboard, of course before I can explain the problem that needed solving , some back ground. The framework has a built in collaboration API, within the design of the framework "API" has a different practical use from how it is defined by the acronym of the words "application programming interface", in the framework new classes are incorporated or assimilated into the foundations of a common base class. The software engineers among you realize that the hallmark of efficient OO programming is programming that employs the full tools of object orientation provided by a developers chosen language. My framework , has the core OO language of java as its foundation and the use of deep OO concepts like abstract base classes, polymorphism and encapsulation were key concepts that I had to master in order to ensure that the best (in my view) solution to the over all problem of "how do I reduce the code size that will be written by a client programmer in the future by designing the base class objects efficiently now?" Luckily for me, the concepts of object orientation are not new at all, in fact I have visited them in various ways in various areas of my life.

As a young child I spent a great deal of time with pencil and paper in hand, the reason was that I had an insatiable desire to draw what I saw. From the moment that I could hold a writing tool I was drawing, I noticed that I had an ability to render images a bit more accurately than my brothers and sisters and peers. I fed my desire to learn by reading comic books , I marveled at the lines and technique used by my favorite artists as a high school student. I was in love with MC Escher's geometrics and tessellations, I was in awe by the rendering of human and animal tone by Boris Vajello , Frank Frazetta and I was blown away by the effortless American reality expressed by Norman Rockwell. As I got deeper into comic book drawing I mixed the influences of these modern illustrators into my own style.

In my early college days I bought books on anatomy and composition to formerly learn the techniques of illustration and this is where OO was introduced full force. The evolution of a drawing proceeds very much like the solving of an OO programming problem. A problem is identified, aspects of the problem are mapped or abstracted to objects that describe the entity and its attributes (loosely, how it interacts with other entities) the mapping or abstracting stage to me is the most important , as it is here that the full mental introduction is made to the variables that must be manipulated to solve the over all problem. Knowing the objects in short, ensures development of efficient solution since the act of "knowing" involves understanding how those objects "fit" conceptually with one another by their attributes. In drawing the same thing is done, accept where in OO programming the objects are classes with attribute lists and return types, encapsulation keywords or scope specifiers in drawing the objects are the conceptualization of physical items, length, breadth , depth, curvature. No surprise that the principle method employed for learning to draw human figures , for example is rote memorization and drawing of human anatomy. The first task is to know the objects just as it is in OO programming, except in this case "to know" means to understand how a particular physical object will appear from various directions of view or under given modes of light. Once abstraction has been done, the bulk of the solution to the problem is already in place, the rest is simply what I like to call the popcorn, the implementation follows from simply putting the objects with known parameters and attributes together in the ways that they can, their "shape" constrains the final construction...just as the varied shapes of lego blocks constrain the types of objects that can be built from them.

In a drawing , the 'popcorn' involves rendering the objects (appendages, pelvis , thorax, head) according to the constraints of perspective on their "shape" as viewed from a particular vantage point. To people that don't draw this looks like magic, but it is nothing of the sort, it is the application of the vast database of point of view drawings that were done by any diligent illustrator during their "learning" stages. Obviously , once committed to memory the act of recalling these perspectives seems wondrous but it is simply a book keeping task, or rather a book reading task of going to memory, finding the view of an object (head, limb, torso..etc.) based on a desired perspective and then rendering it in place. As one gains facility with this task the act is no longer conscious in fact it becomes refined to the point that a given object can be manipulated in the mind. Objects that have various axis of symmetry can be exploited to rotate, translate or dilate them without having to recall a template, since most objects in the real world have at least one axis of symmetry this allows a great reduction in the number of memory references required and allows even more facile rendering of objects, eventually the skill is refined to the point that the object rendering is first preceded by a symmetry finding step which then allows dynamic manipulation in the mind prior to rendering, this is how entirely new views (which might have never before been rendered to paper in practice or otherwise) can be synthesized on the fly. In this way a new and unique drawing can result .

You are probably still wondering how this is similar to programming, the answer is in that great word abstraction; The object models for the shape of appendages, limbs or other animal body parts come from a collection of experiences of those shapes through previous practice coupled with a real time hunt for symmetries in the objects as viewed in the desired rendering. This is precisely analogous to the view of a solution for a given software problem, replace the physical objects by conceptual ones in the problem space with attributes and return types and you have in essence the same building blocks, the view of the final solution involves understanding the symmetries (again) of the objects and using them to efficiently construct the solution.

This takes me back to the problem I solved today, the issue revolves around providing a guest private message ability to the framework. All conversations have a type attribute, currently there are 3 types , private messages (IM), private conferences , and public conferences adding the private guest option might seem to necessitate a new conversation type. However each true conversation requires a row in a corresponding table to persist the conversation attributes. I wanted to allow Guests to engage a conversation without creating new conversations for each remote guest request which would be resource exhaustive. How to solve ? Rather than actually manage the guest conversation I decided to simply use an existing conversation as a mask or proxy, since all guest conversations are initiated by the remote guest only they have a unique symmetry from the perspective of the collaboration API from all other conversation types. If I associate this symmetry(the unique IP of the remote request) with the guest pm request I can denude the proxy conversation, repopulate its attributes with the unique guest attributes and then allow for designated users to engage the incoming guest requests. This solution to the problem would not have been obvious had I not recognized in a visual sense the symmetry of the guest conversation request when compared to the User requests.

Tomorrow I will continue with the implementation of this solution, I have to resolve the fine details but again that is just the dry pop corn, a little butter and salt brings the fun.