Big Ball of Mud - Revisited!

Big Ball of Mud (more likely Great Balls of Fire that will burn you down!)

I am being mentored by Arun Batchu this semester looking into the exciting world of software architecture and as a self study we should look into this Big ball of Mud. I have read the page before and was also mentioned to me recently by my peer as we were discussing software architecture, Conway's Law, microservices architecture.

 The Big ball of mud page (1999) existed before the Agile Manifesto (2001) was conceived. This means that the observation that the authors wrote were noted before most of the agile processes ever existed but I somehow still observe the same kind of things (now more prevalent) in an agile world.

 Well I guess the forces that they have observed still applies to the software projects of the agile world. How would we consciously prevent ourselves from creating Big balls of mud ? I think that there is no clear answer to that because any software project will be constrained by time and budget. But we can take steps to ensure that at the point that we need to ship our software system it will have a structure that will be good enough to handle changes. Nobody will be able to build the perfect elegant system because nobody can predict the future all we can do today is to make a best guess and often times make a trade off when we make that guess.

What are some of the steps we can take?

1. A complex system with a lot of functionality will definitely need a structure to support its function.
    So if the system is complex enough (however you want to define complexity) you must advocate to the product owners that you are going to need to invest some time and money on the software architecture because it will dictate the pace of the work that is coming. But don't invest too much because you are going to need to create functionality to satisfy business or product owners. As i said in my previous post you can interleave architectural tasks with user stories, think of alternative architectures but not commit to it yet until you have enough information which will be a best fit for the system. You still need define the architecture, know your quality attributes but define the architecture loosely but make it clear as more information comes along to help you decide during the software project.

 2. Re-evaluate and refactor the systems architecture for each iteration
    This will give you a chance to use the learning that you have gained during the iteration and enhance the systems architecture. At this point, you can also prevent deviation from the architecture and its erosion.

 Functionality and feature will always trump good software architecture, there must be discipline to make sure that an evaluation of a software projects architecture during a retrospective (feedback loop at the end of an iteration)

Comments

Post a Comment

Popular posts from this blog

OAuth 1.0a Request Signing and Verification - HMAC-SHA1 - HMAC-SHA256

Image
This is a security project that I did this Fall semester. I was interested in why the OAuth1.0a specification did not require you to sign a request using RSA-SHA1 or any RSA algorithm. The first issue that I found is where would you store your private keys in a javascript client? - as a javascript encrypted file ? from the browser ?  - that will be a security hole if you allow javascript to get private keys. These are some of the questions that got me asking and which I cannot find answers to for now. OAuth 1.0a Signature Signing and Verification using JavaScript Implementation Background The OAuth specification tries to define an API authentication method that allows consumer websites or applications to access protected API Resources from a Service provider. The advantage that OAuth provides is that the user of the website or application is not required to disclose their credentials to their service provider. Another advantage of using OAuth to authenticate is that it does not
Read more

Spark DataFrame - Array[ByteBuffer] - IllegalAurmentException

IllegalArgumentException - ByteBuffer - Spark DataFrame I was processing a several million documents (~ 20 million) in which we need to extract the NLP features using NLP4J, OpenNLP, and WordNet. The combination of the three NL features blows up each record to 11 times its original size. We are using all three because we do not know yet what feature sets will be helpful to us. The original dataset is in parquet files in HDFS (16 partitions). I thought that was convenient just use withColumn and pass a UDF (User Defined Function) on the column where it needs those features. withColumn adds the calculated column back to the DataFrame. So I created the spark job (I am on Spark 1.5.2-cdh5.5.2)for the above, and things started to get nasty. I am blowing up the ByteBuffer array on the in-memory columnar storage. This is the exception that I am getting. There seems to be no reference in my code in this stack trace. java.lang.IllegalArgumentException at java.nio.ByteB
Read more

Gensim Doc2Vec on Spark - a quest to get the right Vector

Ever since I joined the R & D group we have been doing a lot of cool things, like trying IBM Watson (see previous blog entry). Now we are doing a lot of Natural language processing.  We wanted to compare the similarity of two documents. There is this excellent project Gensim ( doc2vec ) that easily allow to you to translate large blocks of text to a fixed length feature vector to make comparisons. Here is the link to the  original paper  from some people from Google that explains the approach. In essence, they wanted to find a representation that will overcome the weaknesses of the bag of words model. The doc2vec approach proves to be a reliable approach to comparing similarities of documents because it takes into consideration the semantics and the order of the word in context. So with that, we wanted to use it for a corpus of 26 million documents. Calculating the doc2vec for 26 million documents is not a small task, so we need to process it in Spark. The problem is that there
Read more