A more detailed look at modularization

Jan 14th, 2009 | By | Category: architecture

Any big application typically has a lot of functionality. It is logical that such a large amount of functionality be distributed across modules with each module becoming a specialist in implementing a given set of functionalities. This thought process has of course been crystalized as the single responsibility principle (SRP) that I have alluded to before in other posts in this blog.  These modules get integrated to form the entire product end to end. The module structure for involved products can get very complex with an intricate set of dependency chains that link one module to another. For instance, let us assume that we have a simple product that takes care of managing savings bank accounts. To implement the functionality of the application, let us say we have two modules customer profile and savings. The customer profile module takes care of setting up customer profiles while the savings module provides the savings bank account functionality. Indeed, it is by colloboration of these two modules that the functionality of the application is implemented.  

Modules must exhibit a high level of cohesion that should co-exist with a well defined set of dependencies. These dependencies take care of establishing well defined points of coupling between this module and other modules.

Modular Design and Re-use

Modularization facilitates re-use. Let us say that in our savings bank application, we decide to pull out the customer profile module and make it available for other applications such as a brokerage account application. Quite clearly, this would not be possible unless:


  • The Customer Profile module has a set of functionality that can be re-used by other applications. This point is obvious. Unless there is functionality that the module makes available for others, why would they want to use it ?
  • The module itself has well defined set of interfaces/classes that it makes available for consumption by the outside world. This means that the module itself makes available a few interfaces and classes that classes outside the module can utilize. Again, unless there is something that we gain from the module, why would someone use the module? Further, this point also indicates that the classes or interfaces available to the outside world must be well defined/established or published. This is to ensure that once another module starts using the features of this module, it only can use the features through a well known set of classes or interfaces. As the Customer Profile module evolves, these published classes and interfaces would only need to  be taken care of from the backward compatibility point of view. The others would be internal to the Customer Profile module and hence can be changed without worrying about backward compatibility with the dependent modules.
  • The module has a set of well laid out dependencies on other modules. (called efferent dependencies or outgoing dependencies) If this module does not exhibit well defined dependencies to other modules then the modules that depend on it (called afferent dependencies or incoming dependencies) would not know what they are getting into!  We should let everyone know what we are depending on so that when others depend on us, they would also know that they would indirectly depend on our dependencies. 



SRP, DRY and Modules 

Single responsibility principle states that each class must implement one responsibility. This would ensure that the design is not changing multiple functionalities just because one class happens to change. This also ensures that the class is set up for maximum re-use. A class implementing one responsibility can be more effectively re-used rather than one which is doing too many things. In short, the more a class does the less re-usable it becomes!

Also, DRY stipulates that given a responsibility there should not be multiple classes implementing it. This is to ensure that a change in responsibility does not percolate through the application changing multiple classes thereby increasing the brittleness of the design.

These concepts would also manifest themselves in module design. Given a “set” of responsibilities, there should just be one module implementing them. Hence all the Customer related responsibilities such as maintaining the Customer, keeping track of the state of the customer (whether they are active etc.) must all be done in the Customer Profile module and not in any other module. Conversely, given a Customer Profile module, it should not be doing unrelated work such as Account Management for instance! 

Modules Implementing Horizontal Requirements

Horizontal requirements are unique in that they have to be absorbed across the board by every module. These need to percolate across the entire application. For instance, how do we handle security? or logging? Doesn’t this concern manifest itself across multiple modules? such as the Customer Profile module and the Savings Account module? 

Horizontal requirements are typically implemented in two parts. Firstly, there would be a module that would implement the core horizontal requirement. Example logging as implemented by log4J, security as implemented by any security framework etc. But there is a second part to this. This is the actual incorporation of the concern in the core module. For instance, how do we incorporate security in the Customer Profile module? We first need to include the security module in the application. Then we need to enable the security features into the Customer Profile module. Here, it is important that we make a distinction between implementing security and incorporating secure features into a particular module. Only those features that are necessary for incorporating security in Customer Profile module should reside in the module itself. This important distinction is necessary to comply with SRP and DRY for modularization. 

Modules and Product Customization

When we customize products for different customers how do we exactly go about doing it? Customization is a fairly common problem in product development. One product is written with an extremely generic feature set. Customers start using it and realize that it should be tweaked a little to cater to the particular nuances of their organization. In fact, in many cases the customers would be using existing products that are already doing specific portions of what our generic product is doing. So they might choose to continue using those products without substituting them with ours. So our product must have some important features that would be outlined below.

Strict Module Isolation 

One module should have the ability to work with different modules for implementing functionality. Example, our Saving Bank module must be able to utilize third party Customer Profile modules. This is a difficult proposition to implement but can be done by clearly identifying dependencies. The Savings Bank module must clearly stipulate what contracts of Customer Profile module it is using so that these contracts can be substituted and implemented by a third party module.

Often, database dependencies are harder to implement in this regard since database queries for instance can assume that both the accounts table and the customers table are co-resident in the same database. These decisions are to be taken consciously keeping both performance and dependency management in mind. Let us take the example of a query that has to return all the accounts of a customer along with the customer name. A typical database query in this regard would look something like “select c.name, a.* from accounts a, customer c where c.id = a.customer_id and [ some other conditions] “. This innocent looking query has the potential to increase the coupling between Customer Profile module and the Savings Account module since it assumes that both the accounts and the customer tables are co-resident in the same database. Further, it knows about the schema of the database of the Customer Profile module. Hence this seemingly innocuous query violates encapsulation and  increases module coupling.

What would be the solution to this problem? Let us look at an approach below.

One approach would be to split the query into two parts. One part is provided by the Savings Account module. This goes and obtains all the accounts that are of interest to us. Then the customer profile module is called to obtain the names of the customers who are the owners for these accounts. This might be a do-able solution except that it is not performing at all. Further, it has the potential to trigger the “n+1” selects problem. For each of the “n” rows obtained we are seeking out customer names from the customer profile module – one query per row returned. This is going to kill performance. This can be optimized a little bit by using the SQL “in clause” to stipulate all the customers whose names we would be interested in retrieving. But that approach also is not fool proof. If we have 1000 unique customers returned by the accounts table how do we put them all in one IN clause. The query would become gigantic and may not even be supported by many databases. 

Further, the customer ID column of the accounts table cannot have any constraints if the customers table is not assumed to be co-resident in the same database. Hence we would have a potential to lose some essential constraints in the database. One approach is to live with that limitation. The other approach is to use a customers table in the database and source it from the Customer s table of the Customer profile module irrespective of which database it exists in. This would mitigate some of the design issues and also alleviate problems of module isolation. All these approaches must be carefully documented to ensure that tight coupling issues are well understood. 

Soft Dependencies & Runtime Discovery

Modules can have two types of dependencies – hard and soft dependencies. The hard dependency is “wired” during compile time whereas the soft dependency is dynamically wired only during runtime. In a well designed application, all hard dependencies are interfaces whereas the soft dependencies are implementations of the interfaces.  Hence if the interfaces and the implementations are co-resident within the same module, then all module dependencies tend to become “hard”- i.e. one module can only be compiled and run in the presence of another module. This results in less than optimal situations because the application has the danger of becoming monolithic. In our example, Savings Account module would necessarily need the Customer Profile module. Thus despite the care that we have taken to ensure well defined dependencies, our efforts got foiled by hard dependencies.

To alleviate this situation, we have to ensure that most dependencies become soft.  The only way to do that is to separate out the interfaces from the implementations into two modules. Hence the customer profile module becomes separated into customer profile interface(cp-interface) module and customer profile implementation(cp-impl) module. The saving account module would have a hard dependency on the cp-interface module but have a soft dependency on the cp-impl module. The cp-interface module  will have to be packaged with the saving account module but the cp-impl module would be discovered during runtime. The discovering mechanism would also have to wire the implementation up with the savings account module during runtime. This is facilitated by frameworks such as  dependency injection frameworks. Hence dependency injection framework (aka IOC – Inversion of Control frameworks) play a prominent role in implementing runtime discovery.

Be Sociable, Share!

 Raja has been blogging in this forum since 2006. He loves a technical discussion any day. He finds life incomplete without a handy whiteboard and a marker.

Tags: ,

Leave Comment