Sunday, January 1, 2017

Book review - Java 8 in Action

Java 8 was released in the first half of 2014, and it's awesome! The most notable Java 8 features are Lambda expressions, Stream API and Date/Time APIs. There are also a lot of other improvements like default methods which allow addition of methods to interfaces without breaking existing implementations and allow multiple inheritance of behaviour, but not state. Annotation on Java Types and repeating annotations were also introduced in Java 8.

Even if I was using some new features of this update for some time, I felt it was time to get a deeper knowledge of Java 8. There are several books already published on this topic. I've chose "Java 8 in Action: Lambdas, streams, and functional-style programming" based on the good ratings and high number of positive reviews at the time searching for a book, and let me say it, I don't regret it! The book was amazing!

Java 8 in Action is easy to read, but it keeps you excited throughout the whole book. The authors have done a really good job explaining the new features of Java 8. Throughout the book, new concepts are followed by small exercises which can help you to validate your understanding of the concept. A learning benefit for you is that each chapter ends with a good summary of what was discussed.

The book is split into 4 parts, first part is called "Fundamentals" and it contains a short introduction about why these new features are relevant to you, as a developer. In this part you'll learn about "Passing code with behavior parameterization" and get introduced to Lambda expressions. Second part is called "Functional-style data processing" in which Streams are introduced, explained and exemplified. You will be taught how to "collect data with streams" and you also get to learn a lot about parallel data processing and performance. Third part is called "Effective Java 8 programming", in this chapter you'll be introduced to Default methods, to Optional as a better alternative to null, some new concurrency utilities and the new Date and Time API. Forth part is called "Beyond Java 8" in which you learn more about Functional Programming and in which there is an interesting chapter called "Blending OOP and FP: comparing Java 8 and Scala", you'll like it!

As you'll read the book you will realize that it is not just about Java 8 features but you will learn a lot of other things as well. I strongly believe that after reading this book you will not only be able to use Java 8 features but you will also become a better programmer.

I really enjoyed reading this book and I strongly recommend it to you as well. Enjoy it!

Saturday, August 27, 2016

Book review: The Pragmatic Programmer - From Journeyman to Master

These pages published in 1999 contain a tremendous amount of wisdom for all programmers who wish to become software craftsmen. The book contains a collection of ideas, observations and recommendations for programmers who aspire to raise the bar of professional software development.

While reading this book you will feel like you are having a conversion with two very skilled software craftsmen, you might agree or disagree with their ideas but you cannot argue their judgment.

The authors did a nice thing introducing each chapter with a small context attached, also when new concept or section is introduced it has references to things to which it relates. Also there are interesting exercices at the end of each section.

When reading about portability concerns you'll find links to dead technologies like CORBA or OS like Windows 2000, this will definitely make you reflect on how fast technology has evolved. You'll also appreciate that while sharing knowledge about various topics, the authors shared some good jokes as well.

The book has eight chapters, some of the most interesting topics discussed are: estimations, there are amazingly useful tips about estimating, debugging. Also things to do before a project starts, and topics about pragmatic projects and pragmatic teams. At some point in the book you'll also find explanations for Liskov Substitution and Demeter’s Law.

As a software developer you'll find that even if this book was written many years ago, the shared wisdom is still relevant today.

Monday, February 29, 2016

Microservice and Monolithic architectures exemplified

We'll use an example of an application to illustrate the differences between Microservice architecture and Monolithic architecture.  Let us take an e-commerce application which has the purpose of selling products online. In general all applications from this category require functionalities for browsing through available products, purchase them and placing orders which are later managed by administrators. And of course most of the e-commerce websites have some content which must be easily editable at any time from an administration dashboard.

Monolithic architecture

In the below diagram it is presented an e-commerce web application built following a monolithic architecture. As seen in the below diagram the data of the web application is stored in a single database. Products, website content, orders placed by customers and inventory information all stored in the same place. The web application has multiple purposes, first and most important to allow customers to browse through products and allow them to buy, secondly managing orders, content and maybe offering customers the possibility to create an account for managing different settings and subscriptions.
In this case the application built on a monolithic architecture offers both a web interface written using HTML, CSS and JavaScript and an Application Programming Interface which can be used by the clients such as Android or IOS smartphone apps.

E-commerce web application built using Monolithic architecture
E-commerce web application built using Monolithic architecture

Scaling for this kind of applications can be done both horizontally and vertically. The later means that the production machine will get its hardware improved by adding additional RAM memory, disk storage or adding a better CPU. Scaling vertically can be achieved by installing the same version of the application on multiple nodes and putting a load balancer between the nodes and the browser clients or API clients.

Microservice architecture

The below diagram shows a possible way to build the described e-commerce web application in a microservice architecture. As seen below the application was split into multiple small services each having its own purpose, managing orders, managing products and managing content. 
Each small service has its own database which contains the generated or used data by that particular service, so basically we do not have a big database but rather multiple small databases. This represents one of the big advantages of microservices because scaling of the database horizontally becomes possible. Of course that a data consistency issue appears when a complex flow, which passes through multiple services, fails somewhere in the middle of the processing. Assuring data consistency requires now a lot of extra effort, something which was previously easily achieved using database transactions.
The earlier mentioned API Gateway stands in front of the clients and incorporates the API offered by all microservices such that the clients know only about that API and are not aware of the other services behind it. This can be compared to the fa├žade design pattern in which a single entry point for the clients is offered. Following this architecture makes it easier to later change the implementation or the API of the services as long as the API of the Gateway still remains the same.
Using Microservice architecture offers more options for scaling the application, for example one can install half of the services on one node and the other half on the other node. One can install each service on a separate node or even install each service twice on two different nodes, but then most probably a load balancer is needed.

E-commerce web application built using Microservice architecture
E-commerce web application built using Microservice architecture

In future posts we'll have look over the advantages and disadvantages of Microservice architecture and over its use in practice.

Wednesday, February 24, 2016

Introduction into Microservice architecture

The purpose of this post and future related ones is to understand which are the advantages and disadvantages of Microservice architecture. This blog post is meant to get you familiar with this topic and represents a first step towards reaching the end goal.
Understanding what microservice architecture is and then identifying the advantages and the disadvantages of this architecture, it is first required to define some concepts like software architecture and software pattern. 
Software architecture refers to the high level structures of a software system, the discipline of creating such structures, and documentation of these structures. These structures are needed to reason about the software system. Each structure comprises software elements, relations among them, and properties of both elements are relations. The architecture of a software system is a metaphor, analogous to the architecture of a building. Wiki
As microservices architecture is an architectural pattern let us have a look over its definition. An architectural pattern is a general, reusable solution, to a commonly occurring problem in software architecture within a given context. Architectural patterns are similar to software design pattern but have a broader scope. The architectural patterns address various issues in software engineering, such as computer hardware performance, limitations, high availability and minimization of a business risk. Some architectural patterns have been implemented within software frameworks. Wiki
Because the microservice architecture comes in contrast with the monolithic architecture let us shortly review what a monolithic architecture is. 
In software engineering, a monolithic architecture describes an architecture in which the application single-tiered software application, its user interface and data access code are combined into a single program from a single platform. A monolithic application is self-contained, and independent from the other computing applications. The design philosophy is that the application is responsible not just for a particular task, but can perform every step needed to complete a particular function. Martin Fowler article
Today, some personal finance applications are monolithic in the sense that they help the user carry out a complete task, end to end, and are “private data silos” rather than parts of a larger system of applications that work together. Martin Fowler article
Microservice architecture can be defined as an architectural pattern or style in which a software system is developed as a group of smaller services, each running in its own process and communicating with each other using lightweight mechanism such as HTTP. The services should be built around business capabilities and should be independently deployable. The services should be small, highly decoupled and should focus on doing small tasks, facilitating a modular approach for building a system.


The development of monolithic applications becomes slower as the application grows, so does grows the frustration of the developers. Large applications are hard to manage and most often doing a small change takes days to identify the impact and hours to write the code, afterwards it might take few days to pass review sessions and run automated suite of tests. These issues are some of the causes which made developers to welcome the microservice architecture which breaks down things into manageable pieces.
Following this architecture means that a big team can be split into smaller teams organized around microservices, these small teams become autonomous and fully responsible of their developed service. Fear of change will not prevent developers to fix issues or create new features, besides this the efficiency and the development speed is visible increased.
Developers feel more comfortable when managing smaller code bases than managing a monolithic one, this means that creativity is stimulated and development frustration is less likely to appear.
Another great thing which is achieved using this architecture is that developers are kind of forced to develop the application into a modular way; it also allows reusing services over multiple applications.

More about microservices..

In future posts we'll discuss more about Microservice architecture vs Monolithic architecture, advantages and disadvantages of Microservices and putting this architecture to practice.

Saturday, January 30, 2016

StreamInsight components

Streams of data

Just as Microsoft SQL Server was designed to allow developers to manage static data, StreamInsight was designed to work with streams of data. But what does a stream of data me

an? Well a stream of data is a sequence of pieces of information, for each such piece of information a certain time is associated with it. Usually the associated time is the date-time of creation.

Such streams of data can be produced by countless devices which vary from smoke sensors, temperature sensors to smartphones, robots, web applications, hosting servers or trading applications. 


An event can be defined as a basic unit of data processed by the StreamInsight server, each event encapsulates a piece of information thus we can say that a stream contains a sequence of events. Each event consists of two parts, the header and the payload.

The header defines the event kind and temporal properties of the event. All the temporal properties are application-based and supplied by the data source rather than a system time supplied by the StreamInsight server. All the timestamps use the .Net DateTimeOffset data type, also StreamInsight normalizes all times to UTC date-time automatically.

The payload is a .NET data structure which holds the data associated with the event, the fields of the data structure can be defined by the developer. Each field can have a .NET data type e.g. int, float, string etc.


The same way as a fisherman uses a fishing net to catch fish from a river, we can use a StreamInsight query to retrieve relevant information from a stream of data. The results of the query are received incrementally for as long as we need.

One can define a myriad of queries starting from simple ones like selecting all events which fulfill a certain condition, to more complex like selecting certain events which appear in a window of 3 minutes.

The main difference between StreamInsight and a database is that StreamInsight never stores the data, a query is kept active all time while the server is running. Every time a new event appears it triggers a new computation and generates a new result. Of course if we are interested we can store the results of the queries.


Devices which are data producers become sources of data for the StreamInsight application. One can define multiple sources of data which go into the StreamInsight server and against which queries are executed.


We have sources of data which become streams of data and queries which are executed against them, but how can we get our hands on the query results? Well we can define a sink in which StreamInsight will send the result of the defined queries. Also in this case we can define multiple custom sinks, some might represent a conventional database or an user interface in which users can acknowledge the information immediately.

StreamInsight components working together

As seen in the architecture diagram from above, one can define source of events like smartphones, fire sensors, smoke sensors, temperature sensors, server logs or event historical data. This platform allows developers to aggregate all these events by defining LINQ queries, the results of these queries are then passed to any developer defined sink like monitoring devices, monitoring applications or even data warehouses.

Saturday, January 2, 2016


StreamInsight is a platform developed by Microsoft which allows developers to create and deploy complex event processing (CEP) applications. This platform is based on the existing .NET Microsoft platform and it enables developers to implement robust and highly efficient CEP applications. There are a lot of possible event sources, some of the most relevant are:

  • Financial trading applications
  • Web analytics
  • Manufacturing applications
  • Server monitoring applications

One can use this platform to easily create tools to monitor data from multiple sources for meaningful patterns, trends, exceptions and opportunities. Analyzing and correlation can be done incrementally while data is produced (in real time) without storing it first, which translates in to having a low latency application. As a source of events historical data can also be used.

Key Benefits

In the following I will try to talk about the most important features and advantages offered by this platform.

Highly optimized performance and data throughput

StreamInsight supports highly parallel execution of continuous queries over high-speed data because it implements a lightweight streaming architecture. The use of in-memory cache and result computation done incrementally provide an excellent performance with high data throughout and low latency. In StreamInsight all processing is automatically triggered by incoming events based on defined queries. Also the platform provides the functionality for handling out-of-order events and in addition static reference or historical data can be accessed and included in the low-latency analysis.

.NET development environment

Microsoft created the .NET development environment in which programming languages like C#, tools like Visual Studio and services like SQL Server can be easily integrated and used for applications development while still keeping the loose coupling between them. StreamInsight is included in this environment in which one can easily develop fast and robust applications. Developers can write their CEP applications using C#, leveraging the advanced language platform LINQ (Language Integrated Query) to create queries.

Given the fact that there is a large community of the developers already familiar with these Microsoft technologies the cost and time of the development of a CEP application is significantly reduced.

Flexible deployment capability

StreamInsight platform provides two ways of deployment scenarios. First is a fully integrated into the developed application as a hosted (embedded) DLL. The second way is deploying StreamInsight as a stand-alone server with multiple applications and users sharing the server. This means that one can develop multiple, independent, applications which use the same StreamInsight instance. The CEP server runs in a wrapper such as an executable or the server could be packaged as a Windows Service.


StreamInsight allows developers to extend its functionality by giving them the possibility to define their own operators, functions and aggregates to be used in queries and define specific event types against which to run the defined queries.

One of the great things about StreamInsight is that it was designed to seamlessly integrate with any domain specific business logic. This means that the platform does not come with any implemented functionality for specific business sectors but it allows developers to plugin any specific business logic.

CEP Query Visualization and Analysis

Microsoft StreamInsight provides a stand-alone Event Flow Debugger which is a powerful GUI tool that enables visual inspection of a continuous query. One can use this graphical tool to quickly inspect the query tree, replay data processing and perform analysis.

Latest version

Currently the latest version of StreamInsight is 2.3, this was released together with SQL Server 2014 on the first of April 2014. Release 2.3 contains only a licensing update, so any code written against the previous version, 2.1, will still work.

In the future posts about StreamInsight I will present some of its most important components.

Robert Rusu

Saturday, December 26, 2015

Introduction to event stream processing

Today most of the businesses are actively monitoring data streams and application messages in order to detect business events or situations and take time-critical actions. Even though plans are made for businesses, events are the real drivers of the enterprise today because they represent changes in the state of the business.

Unfortunately, as it happened in case of data management in pre-database days, every usage area of business events today tends to build its custom infrastructure to filter, process, aggregate and propagate events.

Building efficient, scalable systems for monitoring and processing events has been a major research interest in recent years. As new technologies rise and existing ones are expanding the sources of relevant events is growing exponentially. A lot of technologies have been proposed, including Data stream management, complex event processing and asynchronous messaging. 

One can observe that all these systems share a common processing model but differ in query language features. Besides, some applications might have different requirements related to the consistency of the data which might translate in tradeoffs between insensitivity to event arrival order and system performance. It is clear that some applications require that events are processed in the order in which they arrive or were created, while others are more concerned with high throughput. If exposed to the user and handled in the system, user can specify the consistency requirements per query and the system would adjust itself at runtime to guarantee consistency and manage system resources.

Event stream processing use case

As an example let us consider a financial services company that actively monitors financial markets, individual trader activity and monitors financial markets. Having a desktop application, a trader can track a moving average of the value of an investment portfolio. From the business perspective it is required that the average is updated continuously as stock updates arrive and trades are confirmed. A second application running on the trading floor would extract events from live news feeds and correlates these events with market indicators to infer market sentiment, impacting automated stock trading programs. 

The query would filter patterns of events, correlated across time and data values. In order to bring value to the business, this application needs to provide the information as soon as possible, late events might result in a retraction. Meanwhile a third application might be running in the compliance office monitors trader activity and customer accounts, to watch for law violations, bad intentioned actions or institution guidelines. This queries might run until the end of the trading day or even until it finished processing all the events from that day. These applications carry out similar computations but differ significantly in their workload, requirements for consistency guarantees and response time.

The example illustrates that most real-world enterprise applications are complex in functionality and might incorporate different technologies that need to be integrated and are required to achieve high accuracy and consistency. In following posts I will write about a solution which can be used for building applications from the area of event stream processing. The solution is a platform developed by Microsoft and it is named StreamInsight.