Building Services Using Functional Language: Why? and How?

Background:
The term “Web services” or simply “services” describes a standardized way of integrating Web-based applications using open standards over an Internet protocol backbone. It is a software function provided at a network address over the Web with the service “always on” as in the concept of utility computing.

Web services follow an architectural style wherein services are provided to the other components by application components through a communication protocol over a network. A system consists of many different services, which can be used in conjunction to provide the functionality of a large software application.

The Problem:
Services are designed using Object-Oriented Programming (OOP) languages like C#/Java and various frameworks have been built around them. OOP paradigm consists of objects that have internal states (e.g. private attributes in Java). These states are manipulated by public interface (e.g. Java’s public methods / attributes). This model maps very well in certain problems where the system consists of objects with internal states. One such domain is User Interface (UI) design. Each UI Object has state, e.g. TextField has string, Radio Button has a Boolean value (selected/unselected), Combo box has a selected option, etc. These states are controlled by an external event – either a user click or programmatically by code.

However, services are reactive in nature. They follow an event-driven architecture in which every external action generates an event that the system responds to User (or another service) would send a request and the service would respond. An event may generate other event(s) as services call each other. Their responses are combined to create the final response for the event. An event, when generated, does not change or mutate over time. In other words, there is no internal state that can be changed. And, hence, OOP-based languages are not the best to implement such a system.

A Deeper Analysis:
Before we get to the right way of implementing such services, let’s understand why we use OOP-based languages in the first place. The architecture of the modern computer system can be traced back to von Neumann architecture developed in 1945, where computers would have memory and programs would manipulate the contents of the memory. Early programming languages, like C and Fortran, were based on what is known as imperative paradigm and were closely related to that architecture. They would model computer memory as a state, and imperative programs would be sequences of instructions to manipulate the memory. For example, you can think about variables as your state (as they represent locations in computer memory), and program statements would manipulate these variables. As software became more and more complex, more abstractions like Structure Programming were added to Imperative languages.

OOP was an extension of imperative paradigm introduced in 1980’s that introduced two new concepts: encapsulation and polymorphism (e.g. C++ added the support for classes to C). The developers adapted to the new paradigm quickly as it helped them in tackling the complexity of the software at that time. As the Internet developed in the 1990’s, applications started being developed for the web as well. The Internet came with its own set of problems like security and portability, which dominant languages of the time, like C/C++, did not address. Hence, new languages like Java and later C# were introduced. These languages ran on “virtual machines”, so the web applications didn’t had to interact with the underlying von Neumann style hardware directly. But, these languages were still based on OOP based paradigm, because it was the “way to do programming” back then, whether you were developing a desktop application or a web service, and so a large ecosystem of developers and tools developed around the new languages.

This gave rise to the current situation: New programming languages for web services followed the OOP paradigm because it was the dominant paradigm of its time and because developers were familiar with it. It was NOT because it was the best way to design these new applications. OOP, which was designed as a way to develop applications on von Neumann architecture style machines with manipulation of states, started being used to develop web services even though web services don’t have states and don’t deal with von Neumann style machines directly. To this day OOP dominates the programming world!

The Solution:
A good paradigm for developing web services is the Functional Programming paradigm.

Functional programming is a style of programming where data is immutable, and functions are the building blocks of programs (as opposed to objects in OOP). Functions in a functional programming language are first class citizens, meaning that they can be treated like any other variables, passed to other functions as parameters, and combined to form other functions called Higher-order functions. It supports Algebraic Data Types (ADT), and a program essentially transforms the data by applying functions successively.

So, the question would be, why is Functional Paradigm so well-suited for Web services? The answer is that it very closely models how web services work. Events (whether user generated or calls from other services) and Responses can be very easily modeled with ADT. Data immutability captures the behavior of these events, since once an event is generated, it does not change or mutate. A service can be thought of as a function that accepts an event (input to a function) and gives back a response (output of a function). A service may call other services, which is equivalent to a function calling other functions. You can create a system using a set of services, which is essentially the same as creating a higher-order function from other functions.

As you can see, the concepts of Functional Programming like ADT, Data Immutability, and higher-order functions capture the behavior of elements of Web services architecture very naturally. Functional paradigm is nicely suited for building web services.

Practical Implementation – Jet.com:
We built our platform using Micro-Services based architecture implemented in F#. See here to see why we choose F# as our language of choice.

Use of a managed functional language, like F#, gave us several benefits, like more readable code as functional languages are more declarative in nature, concise code due to the repeated application of Do Not Repeat (DRY) principle, and avoid null pointer exceptions by using option types, etc.

See here to learn more about how we implemented our services in F#, and to learn more about how we build our platform, click here.

6 comments

  1. Hi, I’m a prospective candidate for Software Engineer for Jet.com. How can I build a stack utilizing F#? What’s the stack that Jet.com uses?

    1. The bulk of backend code is written in the functional style (F#) and our system of record is an event sourced system (essentially a log of all actions on the platform). We use a mix of Kafka, Redis, Elastic Search, Azure SQL, Event Store and Azure Storage for our data stores. About 50% of the platform runs on Linux and the rest Windows (although we move into more containerization, we will see a shift towards Linux).

      Being based on .Net, F# can be used anywhere where you can use other .Net based languages like C# / VB.Net. There should be plenty of references available on the web. Lastly if you are interested in working at Jet.com, Please check out this.

      1. What F# tooling are you using for Linux? Also are you using any specific libraries (like Suave or ASP.NET Core+F#) or just lower level abstractions (your own framework)? Curious!

Leave a Reply

Your email address will not be published. Required fields are marked *