A few weeks back I learned about NanoClaw. NanoClaw is an AI assistant you host yourself. It runs inside a container on your computer, and you chat with it through apps like Telegram or WhatsApp. It works with Claude, so you need a Claude account or an API key. I decided to give it a go. A few commands later it was up and running on my MacBook in a Docker container. It is secure because it has no access to anything on my computer.

I tried the WhatsApp channel first, and the experience was not great. Two things got in the way. The connection runs through a QR code, and the code was hard to scan. Once I got in, it felt like messaging myself. WhatsApp links NanoClaw as a device on my own account, the same way WhatsApp Web does. So its replies landed in my own chat thread, with no separate contact on the other end. A Telegram bot is different. It has its own identity and its own number, so it feels like a real second party. I set it up with Telegram, and I have been using it throughout the day since.

I used OpenClaw a couple of months back. With all the talk around security, I never installed it on my computer. I put it on a DigitalOcean droplet instead, isolated from everything. It was fun to chat with it and use it for research. After a while the excitement fizzled out, and I never used it for anything real.

This time, I think it is going to become an essential agent for me.

Here is why it works for me:

• When something breaks, it uses Claude to debug and fix it.

• It is secure and isolated, with no default access to anything on your computer.

• Chatting with it is easy. I use it to research, set reminders, and more.

• It can mount my project folders, so it works on real files, not copies.

• It connects to outside services, so I run them from chat.

• It is always on and remembers everything, so context and connections never reset.

Take the project folders. All of mine live on my computer. I mounted those folders on the container and gave it write access. It still sees nothing else, only what I choose to open up. Now the Telegram bot can work on those folders as if they were local.

It has git built in. It can make changes and commit them. For push, you connect it to your git account through OneCLI. That step is optional. You can let the bot commit and then push yourself from your terminal.

Connecting a service was the smoothest part. I wanted it to work with my Fizzy boards and lists. From Telegram, I asked it to set up the Fizzy CLI. It installed the CLI and asked for my Personal Access Token. I generated one from my fizzy.do account, and that was it. The bot could now read my boards, work with lists, create cards, and mark them done.

The setup is not for everyone yet. You need to be comfortable with a terminal, and a bit of configuration. If that is not you, a tech-savvy friend can set it up for you. Once it is running, it is quite smooth.

Today I still work mostly on my computer. But it is easy to leave it on, sit in the living room, or go for a walk while NanoClaw keeps taking instructions.

For me the biggest benefit is not working away from my computer. The biggest benefit is that I touch fewer user interfaces. Apart from the terminal on my laptop and Telegram on my phone, I interact less and less with software UIs. Fizzy is one example. I do not open the Fizzy dashboard in a browser and click buttons to manage cards. I do it from the chat. I expect to connect more software to it over time and use all of it without the interface. That makes a lot of sense to me.

And because it never resets, I never start over. I pick up a task where I left it, and the connections I set up still work. That continuity is what makes it feel less like a tool and more like an assistant.

NanoClaw is not the only AI assistant. OpenClaw, Hermes, and others keep getting better. Share your experience with the AI assistants you have tried and how they have changed your life.

AI agents are the current buzzword in the software world, and they’re quickly changing the way we all live and work. By now, we all have interacted with AI, either while using a chatbot, while searching, or even while performing some tasks. As a software developer and writer, large language models like ChatGPT and Claude have completely shaped the way I work. And now with AI agents, our lives are going to further change, and we will be interacting with AI in almost everything we do. AI agents are at the center of this seismic shift. 

The term ‘AI agent’ is still new. There are many definitions, and people often differ on what can be called an AI agent. The term ‘AI agent’ will evolve as more work happens in this field. My goal with this article is to take a slightly broader view of what people mean when they say something is an AI agent. So, let’s get started.

AI Agent - Definition

An AI agent is an intelligent software system designed to complete a goal that requires multiple steps. These goals are tasks that could have been done by a human agent such as a customer support executive, a travel agent, an HR executive, an IT person, or anyone else. It could be a simple task or a complex task. 

This sounds like any automated system, you may say. However, there are differences. You can ask an AI agent to do tasks it is designed for, and it does those for you. This goes well beyond just answering simple questions, and may involve understanding users’ requirements, figuring out what’s involved, making decisions, and interacting with multiple systems. 

AI Agents vs. LLM Chatbots

While both AI agents and Large Language Model (LLM) chatbots are powered by advanced AI, they differ significantly in their approach and design. LLM chatbots, like the ones we commonly interact with, are designed for quick responses. They generate answers based on patterns in their training data, often providing the first plausible response that comes to "mind." This allows for fast, fluid conversations but can sometimes lead to inconsistencies or less thoughtful answers.

In contrast, AI agents are designed to work more methodically. They don't just respond; they plan, reason, and then act. An AI agent will take time to break down a task, consider various approaches, and devise a strategy before taking action. This slower, more deliberate process allows AI agents to handle complex, multi-step tasks that require careful planning and execution. While this approach may not be as instantaneous as a chatbot, it often results in more accurate, comprehensive, and tailored solutions to user requests.

AI Agents - Capabilities

There are three key capabilities of AI agents that differentiate them from any other automation software or a simple chatbot. An AI agent can 1) Reason, 2) Act, and 3) has Memory.

Let’s look at each of these. 

1. Reason

The first is their ability to reason. An AI agent can understand a user’s request, think about it, break it into steps, and even reason about it. This is a crucial step. An AI agent has a large language model in charge which takes the user’s request and controls the agent’s workflow. This could be ChatGPT, or Claude, or any other specialized LLM. As we know, LLMs are good at these kinds of things, especially to understand the user's intent and to break a complex task into smaller, more actionable steps. And if programmed correctly, they can even reason with you or ask for more information.  This is the core of how a problem will be solved. The model will be prompted to come up with a plan and to reason about each step in the process.

2. Act

The second is their ability to act. This is where agents shine and where the actual work happens. Based on the steps to solve a problem, the AI agent will now act upon each step and execute it. This is done using external programs which are referred to as tools in the AI industry. For example, if a step involves checking the weather in an area, the AI agent may fetch that information from a third-party weather API. This weather API is referred to as a tool. 

An example of a tool is search, such as a program that helps search the web or a database for some information. Another example may be a calculator that will perform some math as required. Yet another could be a custom piece of code that may do something such as interacting with a transactional database, or perform some other operation. A tool could also be another language model or an API that does some specialized task such as translation. The model in charge of the AI agent will orchestrate these tools in terms of when to call them, how to call them, or even which tools to use and which ones not to while performing a certain task. There are endless possibilities for creating and using these tools.   

3. Memory

The third capability is that they have memory. They use memory to remember past conversations from users and access this memory and learn from it while performing tasks, which makes the process even smoother. This allows AI agents to provide more personalized and context-aware responses over time. 

Examples of AI Agents 

AI Travel Agent

An AI travel planning agent could streamline your vacation planning by analyzing your calendar and travel history to suggest suitable dates and destinations. It would recommend flights, accommodations, and activities based on your preferences, make bookings, and add the itinerary to your calendar. The AI could also compile necessary documents, monitor for changes, and provide pre-trip information like weather forecasts and packing suggestions. While ambitious, this level of AI assistance in travel planning may soon be achievable.

AI Email Assistant

An AI email assistant could help a professional by drafting, proofreading, and organizing emails. It would analyze your writing style and past correspondence to compose messages for you, suggest responses to incoming emails, and even schedule follow-ups. The AI could prioritize your inbox, flag important messages, and summarize long email threads. It would also help maintain a consistent tone across your communications, offer language translations, and ensure emails are free from grammatical errors. Such an AI agent could significantly boost productivity and improve the quality of your communication.

Copilot vs Autonomous Agents

AI agents can operate autonomously or alongside other agents and human users. With varying degrees of autonomy, we have different labels for these agents. 

Copilots are a type of AI agents that work alongside users rather than operating independently. Copilots provide suggestions and recommendations to assist users in completing tasks. An autonomous agent, on the other hand, would work more independently, making decisions and taking actions with minimal human intervention.

For example, an AI coding copilot could suggest code snippets, functions, or algorithms while the developer writes code. It could search the project's codebase, offer optimizations, and explain complex code. The AI might autocomplete lines, propose variable names, or generate comments. Developers can accept, modify, or reject these suggestions, enhancing productivity while maintaining control over their code.

An autonomous coding agent could take development a step further by independently generating entire code blocks or features from high-level requirements, analyzing architecture, and integrating modules seamlessly. It might proactively refactor code, handle edge cases, and follow best practices. Developers guide the agent by setting goals, reviewing outputs, and fine-tuning implementations, enhancing productivity while ensuring code quality.

Conclusion

In conclusion, AI agents are transforming how we interact with technology, shifting from simple tools to proactive partners. To make them truly effective, we'll likely need multiple AI models working together, each specializing in different tasks. As AI continues to evolve, these agents will become even more capable, taking on more responsibilities to help us in our daily lives.

This process is what’s referred to as authentication. It's a method for an app or service to verify your identity, ensuring that you, and only you, get access to your personal information.

Why Do We Need Authentication?

Access to all my data in an app must be protected and only I should be able to access it. Apps ensure this through authentication. The security and protection of your data is the most important task of authentication. Another area where authentication enhances an app is through customization and personalization.

Most apps where you log in are customized for you. Twitter has your tweets, followers, and direct messages. Gmail contains your emails. Dropbox stores your files. When you want to access your emails, you log in to Gmail. Gmail authenticates you as yourself, then shows you your emails and allows you to send emails as yourself. Similarly, Dropbox requires you to log in to give you access to your files. Without an account or log in, none of this would be feasible, would it?

When you create an account in an app, they store this information in a database, and then all other data is associated with this account. For example, in their database, they might have a table for all users. In that table, there would be a record for the user ‘manish’ containing user-related information, such as username, email, password, etc. Then there may be another table containing Tweets. Each tweet will have a user account associated with it. So, when I log in, Twitter authenticates me as ‘manish’ and then queries different database tables to fetch various data such as tweets, followers, etc. That’s how I, and everyone else, get a personalized Twitter experience based on data in our accounts.

How Does Authentication Work?

Let’s look at the most common form of authentication, that is, username and password. Here are the steps. (Note that these steps are oversimplified for the sake of brevity)

1. When you create an account in an app, the username, password, and other information you entered gets stored in a database.

2. When you log in, you enter the username and password. In the backend, the app compares this username and password with the stored version. If it’s a match, it identifies you as the owner of the account and logs you in. If it’s a mismatch, it rejects your identity and asks you to try again.

It’s as simple as this. But there are many nuances. Let’s look at some of them.

Understanding Sessions, Cookies, and Tokens

Once you’re logged in to an app, you generally don’t need to log in again when you navigate between pages, or when you return after a few days. For some apps like banking apps, you should be required to log in often for security reasons. However, some apps like social media apps can keep you logged in for months. This is good because logging in every time can be annoying. This convenience is made possible using what developers call sessions and cookies. Frankly, I don’t know why they’re called cookies. Let’s understand how this works.

Sessions

Think of a session as a unique conversation between you and the app. When you log in, the app creates a session on its server. This session is like a record of your interaction with the app, keeping track of your data as you navigate. The session is stored on the server side, meaning the website you’re using keeps track of it. 

Cookies

Now, how does the app remember that you're in an ongoing session? That's where cookies come in. When a session is created, the app sends a cookie to your browser. Cookies are small bits of data that the app stores in your browser. This cookie contains a unique session identifier (ID). It's not your personal information or password, just a random string of characters. 

The Handshake

Every time you make a request to the app (like clicking a link or refreshing a page), your browser sends this cookie back to the app. The app reads the session ID from the cookie, compares with with the session record, and says, "Okay, I remember this session; it belongs to this user." This way, you remain logged in and don't have to enter your credentials every time. Cookies can be set to expire after a certain time, which is why sometimes you have to log in again after a while. 

When something is not working as you expect, your developers or customer service staff of a product will tell you to clear your browser cache. What you are doing here is deleting cookies along with other data stored in the browser. And sometimes doing this magically solves the problem. Now you know why!

Tokens

An alternative approach to storing server sessions is the use of web tokens. When you log in to an app, instead of creating a server session, the app generates a special code known as a web token.

This web token is stored in your browser, somewhat akin to how cookies function. Each time you interact with the app, your browser sends this token back to the server. The token acts as a credential, verifying your identity and permissions.

The key difference between web tokens and server sessions lies in how they handle user data. Server sessions store user data on the server. Web tokens, on the other hand, contain the user information within the token itself. The token itself includes details about your identity, permissions, and any other relevant data. Web tokens have increasingly gained popularity due to their enhanced security, greater flexibility, and superior performance. 

Types of Authentication

Authentication is a big area, and companies keep working on newer, easier,  faster, and more secure ways to provide authentication. Below are some of the most common types:

Single Sign-On (SSO) and Social Logins

The concept of Single Sign-On (SSO) originated in the enterprise sector, allowing users to access multiple applications with one set of login credentials. This approach is logical for employees; once logged into one application, it makes complete sense to be automatically logged into all connected applications using the same session.

Social logins extend the convenience of SSO to the general public. Many companies, including Facebook, Google, and Apple, offer services that let you use their accounts to log into other apps. On many apps, you'll see options like ‘Login with Google’ or ‘Login with Apple.’ This integration is quite practical. 

For instance, consider you have a Google account with the email 'alex@gmail.com' and have used this email to create a Pinterest account. While you can log into Pinterest using a username and password, Pinterest also enables login via Google. Clicking on the ‘Sign in with Google’ button redirects you to Google, where you authenticate with your Google account. Once Google successfully verifies your identity, you're redirected back to Pinterest. Pinterest recognizes that Google has authenticated you with this email, confirming your identity. Although this process might sound intricate, it significantly simplifies the login experience. Pinterest will still maintain a record of your account in their database, linked to your Google-authenticated email.

Passwordless Login Methods

Magic Links: Instead of entering a password, users receive a unique, one-time-use link via email or SMS. Clicking this link logs them in. Magic links are secure and eliminate the need for users to remember passwords.

Authenticator Apps: Apps like Google Authenticator or Authy generate time-based one-time passwords (TOTPs). Users enter these codes to log in, adding an extra layer of security beyond traditional passwords.

Biometric Authentication

This method uses unique biological traits for verification, such as fingerprints, facial recognition, or iris scans. Biometrics are highly secure due to the uniqueness of these traits. They are commonly used in smartphones and high-security environments.

Multifactor Authentication (MFA)

Multifactor Authentication (MFA) is a way of adding an extra layer of security to the login process. It adds another step to your login process for better security. Normally, you just need a password (something you know) to log in. With MFA, you also need a second thing, like a code sent to your phone (something you have) or your fingerprint/face (something you are). This way, if someone guesses your password, they still can't get into your account unless they have your phone or your fingerprint. By requiring multiple forms of evidence, MFA significantly enhances security, making unauthorized access much harder. For instance, even if a hacker obtains your password, they still can't access your account without also having your phone or biometric information.

Companies in the Authentication Business

Authentication is a billion-dollar industry, with various companies offering different services to keep your online accounts secure. For example, Google and Microsoft provide two-factor authentication, requiring both your password and a code sent to your phone for added security. Okta and Auth0 specialize in business-oriented solutions like Single Sign-On, which enables employees to access multiple applications using just one set of credentials. Additionally, Apple's Face ID technology allows you to use your face as a password for a seamless and secure login experience. Companies like Magic.link are pioneering passwordless options, enabling users to log in through a link sent to their email or a prompt on their phone, thus eliminating the need for traditional passwords.

I hope this article has helped clarify the concept of authentication and given you insight into the wide range of services available in the authentication landscape. Feel free to share your thoughts in the comments below.

Important update: I've transitioned this newsletter to Substack and rebranded it from 'Learn Kraft' to 'Modern Software' to better align with its core mission. Rest assured, the essence remains the same—I will continue to distill complex tech concepts into easily digestible insights for you. Now, let's dive into today's article.

As we all know, data is the new gold. Marketers rely on data to gain the necessary insights to understand consumer behavior, preferences, and needs. Without data, creating a successful marketing strategy or campaign in today's business world would be quite challenging. Whether figuring out what your audience likes, the best time to reach them, or what keeps them loyal, it’s data that helps us make informed decisions about all marketing activities.

However, not all data is created equal. You will often hear about first-party, second-party, and third-party data. Let's break down what these terms mean and why they are important.

First Party Data

First-party data refers to the data that a company collects directly from its users, without any intermediaries. These users could be your audience, customers, prospects, etc. First-party data can be collected from various sources such as visits to your website, survey forms, social media interactions, or product purchases. For this reason, first-party data is considered to be the most accurate and reliable. Companies can use this data to gain insights into their customers' behavior, preferences, interests, and needs, which can help them create better marketing campaigns and strategies.

Example: A coffee chain like Starbucks uses its customer order data and customer loyalty app to track what types of beverages are most popular among its regulars. It can use this data to innovate and create offers for its customers.

Second Party Data

Second-party data is essentially another organization's first-party data that you can use for your marketing efforts. It is acquired directly from a source that has a particular closeness or relevance to your target audience, making it quite reliable.

Example: An organic food grocery chain and a fitness gym chain can share their first-party customer data with each other. The grocery chain could share data about what types of organic foods and supplements are most popular among its customers. The gym chain could share data about the workout preferences and frequency of gym visits of its members. Both businesses use this second-party data to create targeted promotions for their customers.

Third Party Data

Third-party data is collected by entities that have no direct relationship with the user whose data is being collected. This data is often aggregated from multiple sources and sold to businesses by data vendors for use in targeted advertising, customer segmentation, and other marketing strategies.

Third-party data includes various types of information such as demographics, web browsing habits, credit scores, social media engagement, location data, purchase history, and market research data. While it provides a broader reach, third-party data may not always be as accurate or reliable as first- or second-party data.

Example: A clothing brand can use demographic information bought from a data vendor to identify potential customers for its new line of sportswear.

Examples - How Different Companies Use Data

Retail Stores: Use first-party data to personalize in-store experiences based on online shopping behavior. They may also use third-party data to expand their customer base through targeted advertising.

Streaming Services: Use first-party data to recommend shows and may use second-party data from social media companies to understand broader entertainment preferences.

Travel Agencies: Use first-party data to understand travel preferences and offer customized packages. They may also buy third-party data to identify new prospects who are frequent travelers.

Automakers: Use first-party data to improve their products and services, while third-party data helps them identify new market opportunities and develop effective marketing campaigns.

In summary, first-party data is direct and reliable, second-party data offers comparable quality, and third-party data provides a broader scope to reach a larger audience. By understanding the differences between these data sets, marketers can make more informed decisions and create meaningful campaigns.

Artificial Intelligence is a type of software system, that, like any other software takes an input and returns an output. Think of it as a calculator. In a calculator, you input some numbers and you get an output. It’s the same with artificial intelligence software, the difference is that the inputs and outputs are way more flexible. For example, when you use software like ChatGPT, you can ask a question in natural language, and it answers your question in very natural-sounding language. Both inputs and outputs are very flexible. So, how does AI software do it? It all happens using a technique called neural networks. This technique was pioneered decades ago, however, it has become so popular only now because of the advancement in technology and computing power.

These AI models can do a variety of things. For example, image classification, where they can identify entities in a photo. Another example is language translation, where they can instantly convert text from one language to another. Yet another example is speech recognition, where they can transcribe spoken words into text.

What is a Neural Network?

Our brains are made up of cells called neurons. All these neurons are interconnected with each other to form a network that stores information and performs various tasks. The brain takes an input, passes it through this complex network, and produces an output. This is the biological neural network of the brain.

A neural network in the digital world is a computational model which is inspired by the brain and is foundational to how artificial intelligence works.

Building Blocks of a Neural Network

At the heart of a neural network is a neuron (or a node) which is a simple processing unit. It takes inputs from other nodes, or from an external source, performs some computations on the inputs, and produces an output.

These neurons are organized in layers. The first layer is the input layer that receives inputs from outside, such as a dataset. Each neuron in the input layer is a feature in the dataset. After the input layer, we have one or more hidden layers that take the inputs and perform calculations on our data. Then we have the output layer in which we get the final prediction or the result from our given inputs.

Every neuron in a layer is connected to all neurons in the previous layer, forming a densely interconnected network. The neurons and the connections between them (represented as lines) have associated values, such as data inputs and weights. These values are essential for the computations carried out by the network.

Neural Network - Example

Let's take an example of a neural network that classifies an animal as a mammal, fish or bird. We provide it with a picture of an animal, and as an output, it classifies the picture as a mammal, fish or bird. Let’s look at what goes on in the three layers.

Input Layer

In the input layer, each neuron will represent a feature of the animal. Some example features are:

• Body temperature: Either "warm-blooded" or "cold-blooded" - e.g., for a domestic cat: warm-blooded

• Number of legs: e.g., for a domestic cat: 4

• Has fur or hair: e.g., for a domestic cat: Yes (1)

• Can fly: e.g., for a domestic cat: No (0)

• Average lifespan: e.g., for a domestic cat: 15 years

• Average adult weight: e.g., for a domestic cat: 4.5 kilograms

These are just some examples. There could be many more features.

Hidden Layer

Here the neural network could have one or more hidden layers, with different numbers of neurons, each taking in the weighted sum of inputs from the preceding layer. These neurons are designed to activate or respond based on the strength of these inputs. These neurons are designed to recognize different patterns and combinations of features. For example, some might prioritize traits common to mammals or birds, while others might emphasize the size and lifespan of the animal. Another neuron might consider all input features to comprehensively understand the animal's characteristics.

Output layer

The output layer in this neural network contains three neurons: one each for mammals, fish, and birds. Each neuron produces a value indicating the likelihood of the animal belonging to its respective category. For instance, the value from neuron 1 indicates the probability of the animal being a mammal, neuron 2's value represents the likelihood of it being a fish, and neuron 3's value pertains to the chance of it being a bird.

The highest value determines the network’s final classification. If we feed the picture of a domestic cat to this neural network, then ideally the neuron representing mammals in the output layer will have the highest value.

Neural Networks Can Learn

But how are neural networks able to perform this classification? They can do so because of their ability to learn patterns and relationships from data. They are continuously learning and improving themselves through a process called training. Training involves feeding the neural network a large dataset, such as pictures of animals, so it can learn and recognize patterns. During training, these weights are adjusted to improve the performance of the neural network. We will learn about the concept of model training in a separate article.

I hope you enjoyed this article. Now, you're one step closer to understanding the world of AI and neural networks.

Companies building products want feedback from their customers. In fact, customer feedback is the most important factor for a company to build a useful and successful product. If customers don’t like a feature or design of something, it will be in your favour to work on the problem. This is true not just for physical products, but also for digital products like e-commerce websites, social networks and software products.

In the digital landscape, one of the most effective strategies to gain insights into what customers want is A/B testing.

What is A/B Testing?

A/B testing is a technique where a company shows two slightly different versions of a part of the product to users and then measures which one performs better. Basically, it is a race between two versions, A and B, where the winner is decided based on the relevant metrics such as improved conversion, more time spent on the product, etc.

A/B test is a type of controlled experiment. A is the existing version and B is the version that contains the changes. The users are split into two groups. One group is shown version A and the other group is shown version B. And then the performance of each group is measured to see which one performed better. If Group B performs significantly better than Group A, then version B is implemented as the new standard. This would mean rolling out version B to all users. Essentially, version B becomes the new version A.

Where is A/B Testing Applied?

Nowadays A/B testing is applied to almost all areas in digital business. People create A/B tests for a variety of things such as the color of the buy button, email subject lines, ads, and testing new software features. Sometimes the color of the buy button can have an impact on the actual conversions. An A/B test can reveal this. Similarly, one email subject line can have a better open rate than the other.

Typically when people talk about A/B testing, they are referring to digital marketing. But it is equally true for software also. In the software world, it is more commonly referred to as feature flagging and experimentation.

A/B Testing in Software

In a software company, the engineering and product teams are primarily responsible for developing new features. But how do they determine which ones to build and where to focus their resources? One effective strategy is experimentation. In this approach, the team first designs a scaled-down version of the new feature, which is then rolled out to a small segment of their user base. The impact of this feature on the chosen group is carefully measured. If the response is positive, the team proceeds to develop the feature fully and release it to all users. However, if the impact is negative or insignificant, they take the experiment as a learning opportunity. Depending on the insights gained, they may revise and retry the feature, or they could decide to abandon it and work on a different experiment.

Experimentation Example

Imagine you’re a product manager at a SaaS company providing CRM software. You’ve received feedback that customers are not very satisfied with the reporting features. You decide to improve the reporting features by adding new reports and more granularity. Instead of creating a big plan, you start by building a small thing, like a new daily sales report. You select a small user base and launch this new report to this selected user group. The rest of the users will not see this new report.

After running the experiment for some time, you compare the engagement levels and feedback from these two user groups. This could include things like time spent on checking reports, frequency of use, and feedback received. Based on the result of this experiment, the product manager can decide to roll out the new report to all users and further plan to work on the overall reporting feature.

Important Considerations

To run a successful A/B test, there are a few important considerations.

• User Selection: The way you select the small user group for testing your new feature is very important. You may select these users randomly, or you might have a selection criterion that's relevant to the feature. Once you've established a selection criterion, you'll need to implement it in your code to ensure that only the selected users have access to this new feature. This implementation is done through a technique called feature flagging. Some companies build this directly into their product, while others use third-party software like LaunchDarkly for this purpose.

• Measurement: Just showing the new feature to the selected group is not enough. You also need the capability to measure the impact of this experiment through the data you gather. For instance, for an e-commerce company, the metric could be increased conversion rates. For a social network, it could be increased user engagement. For a SaaS company, it might be higher product usage. Measuring and reporting such impact is a lot of engineering work.

• Test Duration and Sample Size: It's important to run the A/B test for an adequate duration and on a substantial enough sample size. This helps minimize the impact of anomalies and ensures the credibility of the results obtained from the test.

• Evaluate Performance (Statistical Significance): Once you've started showcasing the new feature to a select user base and you're tracking the appropriate metrics, the next step is to evaluate the performance of your experiment. This evaluation requires some statistical analysis to determine whether the differences observed are statistically significant or merely due to chance. This step ensures the validity of your results and helps avoid false positives.

A/B Testing Examples from Leading Companies

Some of the big tech companies are known to perform multiple A/B tests. Here are a few examples.

• Google: This is one of the most cited examples of A/B testing. Google wasn’t able to decide which shade of blue to use in their ad links. They decided to A/B tested 41 different colours. This was a successful test as a result of the winning shade, their yearly income increased by an estimated $200 million!

• Airbnb: Airbnb ran A/B tests on their search result layouts. They tested a more prominent design of the price-per-night feature by moving it from the bottom to the results to the top. This resulted in higher user engagement.

• Uber: Uber performed an A/B test on their surge pricing feature. They wanted to test if they should display the surge price directly or show it as a multiplier or normal price (like 1.2x). The test results showed that people were more likely to accept surge pricing when the prices were shown directly rather than as a multiplier.

• Booking.com: Booking.com used A/B testing to decide whether showing hotel prices with or without tax led to higher conversion rates. They found out that showing the final price (including tax) from the start resulted in a more positive user experience and increased bookings.

Note that all these companies run A/B tests regularly on a variety of features. These are just a few select examples.

I hope this article has offered you a clearer understanding of A/B testing. Until next time!

If you work in a tech company, API is a term that you will hear about frequently in your workplace. This article is all about APIs.

API - What is it?

An API, or Application Programming Interface, is a piece of code that takes some inputs and gives you an output. An API is the primary way in which different software applications interact with each other and exchange information. That’s why the name. It’s the programmatic way for applications to interact with each other.

Let’s understand this with the help of a few examples.

Google Maps API

We all use google maps. Review our earlier article about backend and frontend, explaining how all software has a frontend and a backend.

In the case of Google Maps, the Mobile app and the Website are the two frontends. When you enter an address, it shows you the location on the map. What actually happens is that when you input an address, the frontend sends a request with the address as input to the Google Maps API. The API is a part of the Google Maps backend, which is waiting for the requests. Once it receives the request, it processes the request, runs all the magic it has to, fetches the coordinates (Lat and Long) and sends these coordinates as output to the frontend. The frontend then displays these coordinates on the map.

This sending a request is known as ‘Calling an API’. Along with the coordinates, the API could also return some other useful information that we see on the screen. Or this additional information could be coming from another API call made by the frontend. It all depends on how the development team decides to structure their APIs.

As we mentioned initially, each API is a piece of code that takes an input, performs some actions, and returns an output. Each such function is called through what we call an API endpoint. So, the API endpoint that we described above could be called ‘/getcoordinates’ or something like this and its job is to receive addresses and return coordinates. Google Maps calls it Geocoding API. Similarly, everything that you do in Google Maps will be interfacing with a bunch of APIs in the backend.

Some more examples of Google Maps APIs:

• Places API: This API provides information about places, which are defined as establishments, geographic locations, or prominent points of interest. When you search for restaurants near me, this API is at work to get the information.

• Directions API: This API provides directions between locations. It can give step-by-step instructions for various modes of transportation, like driving, walking, or public transit. This is what we use when we are travelling to new destinations.

• Traffic Data API: As you can guess, this is the API that adds traffic data to Google Maps when you are travelling.

Public vs. Private APIs

At this point, it’s important to make one important distinction. An API can be a private or a public API.

In the example we saw above, it’s Google’s own front-end apps that are fetching and using the data from Google’s own backend/APIs. This is an internal usage of the API. Google may have 100s of APIs to create the full user experience on the Google Maps app.

Google may choose to make some of these API endpoints public so that other app developers can use these public APIs in their own apps. They will keep the other APIs private only for their own internal use.

Uber App’s Use of Google Maps

Uber is another app we all use regularly to book rides. When we book a ride, a few API calls are made behind the scene to fulfil your request.

Internal APIs

Many of the API calls made in Uber’s app will be to their own backend and internal APIs. Some examples are:

• Authenticating the user: The user logs in and the API verifies your credentials

• Displaying available cars around you: To display available cars around you, Uber would call its own internal API to fetch data about nearby drivers.

• Calculating fare and ETA: Uber would call its internal APIs to calculate the estimated fare and time of arrival based on factors like distance, travel time, car type, and current demand.

• Finding the driver: You enter the pickup and destination address, and the API finds the driver for you.

This is just an educated estimation of how their APIs might work. The actual functionality will be proprietary.

Public APIs

Along with these internal API calls, Uber also makes use of some public APIs in their app.

• Maps: Uber doesn't have its map. Instead, it uses the API of Google Maps to show you the map within the Uber app. This is what a public API does. It allows Uber to use features from Google Maps, making your experience as a user seamless. Google Maps Geolocation API is used to determine and display your current location on the map. Uber may also use Google Maps Directions API to fetch the best path from your current location to your destination.

• Payments: If you decide to book the ride, and once the trip is completed, Uber would communicate with a Payment API (like Stripe, PayPal, or others) to handle the transaction.

These are just some of the potential API calls that could be made. In reality, modern applications like Uber might make dozens or even hundreds of API calls for various tasks during the process of booking and completing a ride.

An interesting point to note is that Uber also has some public APIs which allow other developers to use Uber’s API to integrate some of their data and services into their app. But not all Uber APIs will be public. Only some will be.

To conclude, APIs are a crucial part of all modern software. They enable different software applications to communicate and share data and functionalities seamlessly.

There is a positive buzz about DevOps among Tech companies, and not without reason. What is DevOps? DevOps is the new and improved way for tech companies to build and deliver software.

Dev and Ops - The traditional way

In a traditional setup, there are two teams that are responsible for building and delivering software to customers - the developers (​​who write the code) and the IT operations team (who deploy and maintain that code in a production environment). And these teams have worked in silos and haven’t played well with each other. Since the teams worked independently, it often led to yawning gaps and deep conflict between the two departments.

DevOps - The new way

DevOps is a combination of two words: Development (Dev) and Operations (Ops). DevOps is the modern approach to help software developers and IT operations work better and faster, together. It is a shift in the working philosophy between the two areas. DevOps signals a newer era of efficient collaboration.

In companies that follow DevOps practices, the developers and operations guys work together from the beginning. The emphasis is on collaboration between the teams. This means that people in both teams are concerned about each other's work. The developers are concerned about how the code is deployed to servers and the operations are concerned about the development.

Benefits of DevOps

When the teams collaborate in this way, it results in many benefits. It reduces the development cycle. The bug fixes and new features built by the dev team are deployed faster and more smoothly, sometimes multiple times a day. DevOps relies heavily on automation. Since the teams are collaborating, many things can be automated. For example, changes in code can be tested automated and made ready for deployment. This is a part of the Continuous Integration (CI) pipeline.

Continuous Integration is the practice of regularly merging code changes from all developers on a team into a main codebase. This process happens frequently, often multiple times per day, and it's accompanied by automated builds and tests to ensure that new changes don't break existing functionality.

Similarly, many parts of deployment can also be automated. This is commonly referred to as Continuous Deployment (CD). This allows for faster, more frequent deployments.

In Continuous Deployment, every change that passes all stages of your production pipeline is released to your users automatically. This is achieved through rigorous automated testing that ensures that you're only deploying quality code.

The DevOps Cycle

The DevOps cycle is a set of stages that software goes through during development. Since DevOps is a philosophy and a set of practices, these stages are not set in stone and can be different in different companies. The DevOps stages are represented by a very interesting symbol called Infinity Loop, which emphasizes that the process of software development and delivery is ongoing and cyclical, rather than linear.

Let’s briefly look at these stages:

• Plan: In stage involves planning new features and enhancements, creating specifications, and allocating resources for executing these changes.

• Code: In this stage, the developers write code for implementing the new features and improvements.

• Build: In this stage, the new code is prepared for production and is transformed into a form that can be deployed to servers or delivered to users. This process often involves the use of Continuous Integration (CI) tools that automate compiling the code, resolving dependencies, and packaging all components. For instance, for a mobile app, CI would include creating a new version of the app. For a web app, it's about making a deployable format that can be executed on a server. CI ensures this build process is consistent and errors are caught early.

• Test: In this stage, the new version of the software is tested to make sure it works correctly and meets the project's requirements.

• Release: If the tests are successful, the new version of the software is prepared for release. This often involves uploading the new version in a staging environment that closely resembles the live environment.

• Deploy: The new version of the software is deployed to the live environment, replacing the old version. Deployment is often automated in a process called continuous deployment.

• Operate: The software is run in the live environment, where it is used by end users.

• Monitor: The operation of the software is continuously monitored to ensure it is working correctly and efficiently. Any problems are reported back to the team.

Common Misconceptions

Let’s look at some common doubts about DevOps.

Q: Is DevOps just about tools and technology?

A: No, DevOps isn't just about tools. While tools play an important role, DevOps is a cultural shift focusing on team collaboration.

Q: Is DevOps only for tech companies or startups?

A: Not at all! Any organization that creates software can benefit from DevOps, from banks and healthcare providers to government agencies.

Q: Does implementing DevOps mean we get rid of our operations team?

A: No way! DevOps is about getting developers and operations to work together, not removing operations. Both roles remain critical, but they collaborate more to streamline the process of delivering software.

Q: Is DevOps a job role or a team within a company?

A: DevOps is more of a culture or a way of working, rather than a specific role or team. Although some firms have a 'DevOps team', it's mainly about embracing the DevOps values of teamwork and automation throughout all development and operations tasks.

To conclude, DevOps is a way of working that emphasizes collaboration between the teams that create software and the teams that ensure the software is delivered and works well. It promotes quick feedback, constant improvement, and automation, aimed at delivering software faster, with higher quality and reliability.

Companies are collecting more and more data every day, which comes in all formats and is not necessarily well structured. To store these vast amounts of data, companies are building data lakes. In this article, let's understand what data lakes are.

Before we get to the concept of a data lake, let’s first make sure we understand a few concepts about how companies store data.

Production Database

For any application, such as accounting software, or HR software, all the data is stored in a database. This is the core database that’s required for storing the essential data including things such as user profiles, customers data, orders data, payment transactions data, inventory data and so on. As people use the software, the data is exchanged using simple read-and-write operations. This database is the lifeline of the software. Such a database is referred to as a production database.

Data Warehouse

Most companies need some form of reporting or analytical layer on top of this data. Sometimes reports are quite simple. For example, the management wants to see a report of all the orders that were shipped the previous day. Such reports are easy to generate and can be created by running simple SQL queries on the production database.

Most often, companies need much more complex reporting with data coming from multiple data sources. For example, the company’s sales data will be coming from an eCommerce platform or third-party retailers, their marketing data will be in marketing platforms like Google Ads, Mailchimp, etc, and payment data will come from a payments processor such as Stripe.

In such a scenario, companies create a special place for data, called a data warehouse, where data is aggregated from all these sources, and stored specifically for analytical purposes.

In a data warehouse, companies can store vast amounts of historical data on which they can perform all kinds of analytics such as trend analysis, and forecasting. Unlike production databases, these data warehouses are optimized for complex queries and data analysis. Data warehouses are a key component of Business Intelligence (BI). They provide the processed and organized data that BI tools need to create reports, dashboards, and visualizations.

So, a business has a production database (which handles your day-to-day operational needs) as well as a data warehouse (which is structured and used for reporting and analysis). What comes next? A data lake, you say?

A data lake is like a vast reservoir where you can store all kinds of data, structured or unstructured, for long-term use. Not all the data is used immediately; some is stored for special cases or future analysis.

Why have a Data Lake?

Companies generate two types of data: Structured (like user data, order details, etc.) and Unstructured (log data, website views and clicks, product usage data, etc.). A Data Lake can hold both these types, making it a versatile tool in a company's data stack.

A data lake is a large-scale storage repository that holds raw data in its native format until it is needed. So, the data is just dumped in its raw, unprocessed form, without any transformation or cleansing. Only when specific data is needed, the data is retrieved, processed and analysed as per the need. Data lakes are equipped with robust data governance strategies to ensure data is catalogued, tagged, and meta-data is maintained so that the data remains searchable and usable.

Data Lakes vs. Data Warehouses

Data Lakes are often compared to Data Warehouses. The latter is ideal for structured data with predefined schemas, while the former suits unstructured data without any specific structure. This makes Data Lakes a good fit for special use case data.

The beauty of Data Lakes is the 'Schema-on-Read' concept. This means you can just throw in data without defining its structure beforehand. When you need to use it, that's when you structure it. This grants flexibility but also requires good data hygiene.

Data Lakes & Data Warehouses often work together. Companies typically start with a warehouse & then invest in a lake as unstructured data grows. So it's not about choosing between them but about leveraging them effectively.

Use Cases for Data Lakes

The huge amounts of data stored in data lakes can allow a company to perform a variety of analyses to improve its business. Some areas include big data analytics, artificial intelligence and machine learning, and even real-time analytics. Let’s look at some examples:

• An e-commerce company can store and analyze massive volumes of transactional data, customer behaviour, product reviews, and clickstream data. This can help them identify sales trends, peak shopping periods, and customer preferences.

• A hospital can use a data lake to store a variety of data such as medical records, lab reports, imaging data, and patient histories. They can train machine learning models on this data which can then help them in predicting disease patterns, help in diagnostics, and personalize patient treatment plans.

• Financial institutions can use data lakes to perform real-time analytics by monitoring and analyzing real-time transactions. This can help with identifying fraud in real time and safeguard their customers.

Data Lake Solutions

Companies create data lakes using a variety of solutions. The popular choices for Data Lakes are often the native object storage products of major cloud providers, like AWS’s S3. Some of the solutions are mentioned below:

• Amazon Web Services (AWS) offers S3 coupled with Lake Formation

• Microsoft Azure Data Lake Storage

• Google Cloud Storage

• Cloudera Data Platform for organizations seeking hybrid solutions

• Databricks' Delta Lake stands out with its approach to merging the strengths of data warehouses & lakes

To conclude, data lakes are a powerful, flexible solution for storing vast amounts of raw data, that can be used for varied analytical tasks. They are key to unlocking valuable insights and making informed decisions in today’s data-driven business world.

One of the most frustrating problems on the Internet is when you visit a website but it’s just too slow. You wait for a few seconds and if it’s still not loaded, you lose interest and you are back to Google, searching for something else. Slow loading is one of the biggest causes for visitors to leave the site.

There could be many reasons why a website is slow. It could be a slow server located somewhere very far from the user. It could be some complex code which makes the page load slowly as it takes time to process. Or it could be various assets such as images, videos, stylesheets (CSS files), and javascript files, which all take time to load.

One of the ways companies make their websites load faster is by using Content Delivery Networks, also commonly referred to as CDNs. In this article, let’s learn about what CDNs do and how they make websites fast.

When there is no CDN

Typically, a website lives somewhere on a server located in some part of the world. When a user requests to visit the website, the request is made to the server, and in response, the server sends all the website content which is then shown to the user on their device. Now imagine the user is in India and the server is in New York. What just happened is similar to someone in India asking a delivery truck to travel from New York to India to deliver a package. In terms of data, this is a significant distance to travel, and depending on the quality of the network and other factors, it could take some time. All this makes the website load slowly for the user.

Content delivery networks solve this problem.

How a CDN Works

As the name suggests, a CDN is a content delivery network. The content in the case of a website includes all the things that it loads, including the images, videos, stylesheets, javascript files, and any other files that are needed to load the website.

A CDN stores a copy of your website’s data on multiple servers strategically located around the world such that your website’s data is always close to its users.

When a user requests a website, the CDN steps in, and instead of requesting the website from the origin server, it routes the request to the server in the CDN network that is closest to the user’s location. This makes the trip much shorter for the data and hence the website loads faster for the user.

Benefits of CDNs

Apart from speed, CDNs help websites in many ways:

• Better user experience

• Reduced bounce rates. Users tend to move away from websites that load slowly

• Increased content availability and redundancy. Since your website content is stored on multiple servers, if one goes down, the other server can serve the website to the users.

• Reducing bandwidth costs

• Improved website security

CDN Service Providers

Fortunately, website owners don’t have to set up their own CDNs. Many companies provide top-class CDN services that help you optimize your website’s performance. Some of the leading names in the industry are:

• Akamai

• Cloudflare

• Amazon Cloud Front

• Fastly

• CDN77

• Microsoft Azure CDN

• Google Cloud CDN

Each of these companies has its specialities and differentiating features.

Conclusion

Everyone wants websites to load quickly, work smoothly, and be safe to use. CDNs help make this happen by keeping a copy of the website close to users, no matter where in the world they are.

So, the next time you're browsing the web and a website loads quickly, chances are there's a CDN working hard behind the scenes, to make it reach you faster.

How is it possible for a web app to be serverless? Well, it’s not! Any app or website on the Internet is running on a server located somewhere. The term ‘serverless’ is quite confusing in that sense. But hold on, this is not the first time that some technology is referred to by a name that doesn’t make sense.

Serverless - Definition

The term ‘serverless’ is used to describe the scenario where you don’t have to manage the server at all. In fact, you’re completely oblivious to the server setup and what’s going on inside of it. You don’t have to worry about the hardware, operating system, or even the software running on the server. You just focus on building the software, and when the time comes to show it to the world, you deploy your code on a serverless service and they take care of the rest and make it available for everyone to see and use.

Serverless Functions

One of the most common use cases of serverless is to execute some backend code in response to an event such as an API request. These bits of backend code that we’re executing are called functions. One popular service that allows you to deploy and execute functions is called AWS Lambda functions. Let’s take an example to understand how it works.

1. Let’s say you have a popular website where users can signup or register to use the website. When a user fills out the registration form on your website and clicks the ‘Register’ button, it sends a signal (an event) to the AWS Lambda function.

2. A Lambda function is a piece of code that’s running on AWS. This lambda function’s job is to handle the registration, i.e., to receive the user’s details and add them to a database. The Lambda function is always waiting for the registration events so that it can get to work and execute it. This ’waiting’ is also called listening to events.

3. As soon as the registration page sends this event, the Lambda function receives it and does its work. It takes the user details received and adds them to a database.

4. Once done, the Lambda function goes back to waiting for the next event.

In a traditional setup, you would have set up a server to execute this code and paid a monthly fee for the server. However, with AWS Lambda, you didn’t have to worry about the server at all. And you don’t even have to pay for the whole month. You only pay for the when the function is actually working (processing an event). You pay nothing for the time it's waiting. This makes serverless functions much more cost-effective than having your own servers (server-full setup as some would call it).

Examples of Serverless Functions

Above, we saw an example of how a Lambda function can be sued to perform a backend task, like registering a user. Serverless functions can do a variety of tasks as you can write complex code in these functions to perform tasks for your needs. They can even interact with third-party APIs and services to complete the tasks. Here are a few examples:

• Image Processing: When a user uploads an image to your application, a serverless function can be triggered to perform operations like resizing the image, compressing it, or applying filters.

• Data Transformation: A serverless function can be used to transform data from one format to another. For example, when an Excel file is uploaded, a cloud function is triggered to convert it into a CSV or a JSON file.

• Notifications: We can use the serverless functions to send notifications (email, sms, push) in response to events such as user registration, password change, friend request in a social media app, etc.

• Scheduled Tasks: Serverless functions can be used to perform periodic tasks, such as backups or updating analytics dashboards. Instead of triggering these functions by events in your app, you can trigger them using a timer.

• Real-time File Processing: When a user uploads a file, a serverless function can be used to process the file in real time. For example, extracting text from a PDF file or analyzing an uploaded audio file.

• Data Validation: When data is received (from user input, or third-party services), a serverless function could be used to validate this data before it's processed or stored.

Is Everything Serverless?

If you remember from our article on backend and frontend, there are many parts that go into making a web app run smoothly. There is the frontend, and then there are backend services such as the API, the database and so on. The above example was of a backend task. It’s not necessary that the entire stack of a web app is serverless. You can still have some things running on the server managed by you (like your frontend or database), while the other things (backend tasks) are deployed as serverless functions. Technically, it’s possible to adopt serverless for everything. However, it may not always be productive.

Examples of Serverless Providers

All big cloud providers and many small players provide serverless products for a variety of services. Here I’ve listed a few examples categorized by the type of service.

Cloud Functions

Using these cloud functions, you have the ability to deploy your backend as numerous distinct functions, with each one executing a specific task. This is the most commonly used serverless tech. Some examples:

• AWS Lambda

• GCP Cloud Functions

• Azure Functions

Frontend

While serverless architecture is generally associated with backend processes such as APIs, and data processing, with advancements in web technologies, serverless has found its use in frontend services as well. We can use serverless concepts to serve frontend assets and perform frontend operations. An example of this would be a website built using Nextjs and deployed on Vercel.

Serverless Database

Serverless databases automatically manage the infrastructure for you, allowing you to focus on your data rather than on maintaining and scaling your database. With these databases, there are no servers to provision, or manage, and no software to install, maintain, or operate. They provide automatic scaling, high availability, and high performance. Some examples:

• Amazon DynamoDB: A NoSQL database offered by AWS

• Google Cloud Firestore: A NoSQL database offered by Google Cloud

• Azure Cosmos DB: A multi-model database offered by Microsoft Azure. It supports various data models, including document, graph, key-value, table, and column-family data models.

• AWS Aurora Serverless: A relational serverless database offered by AWS. It supports SQL and PostgreSQL.

A lot of interesting services are coming up in this space. Some examples are Vercel (Static sites and serverless functions), FaunaDB (NoSQL database), BigQuery (Data warehouse), Neon (Serverless Postgres database), and Netlify (hosting and serverless backend services).

I hope this article helped you clarify some doubts about serverless tech. Make sure you watch out for new products and services emerging in the serverless world.

At Learn Kraft, my goal is to help you become knowledgeable about software development. While I’ve been researching the topics to write on, one common concept that needs explaining everywhere is the difference between Frontend and Backend. In this article, I will explain what the terms frontend and backend mean, the difference between the two, how to identify them, their components, and a real-life example. Let’s get started.

Frontend Vs Backend - Analogy

Every application that you use, whether a web app or a mobile app is made up of two parts - frontend and backend. Even this website where you’re reading this article has a frontend and a backend.

The most commonly used analogy for the backend and frontend is that of a restaurant.

• In a restaurant, we have a dining area with beautiful interiors, tables and chairs, a menu, waiters, and most importantly the food that is served to you. These are all the things that you interact with in a restaurant as a customer. In software terminology, this is the frontend of the restaurant - everything that you the user interact with.

• But that’s not the entire story of what makes up a restaurant. A restaurant has a backend too. That’s the area that you as a customer never see but is crucial for running the restaurant. The backend includes the kitchen, the pantry with all the ingredients, the chefs who cook the food, and the utensils. It also includes things that are not tangible, such as the logistics of buying the raw ingredients, and processes for quality control. If anything goes wrong in the backend, it impacts the customer experience in the frontend (sitting in the dining area).

In the context of a web application, the frontend is the user interface - the design, the buttons, the colors - everything the user sees and interacts with directly. On the other hand, the backend includes everything that goes behind the scenes. This includes the server, the database, the APIs, and everything that’s required to provide a good front-end experience to the users.

Identify Frontend and Backend

Let’s take the example of this website and try to identify the frontend and backend parts of it.

Let’s say you arrived at the Learn Kraft website and clicked on the article link that says ‘Cloud Computing’.

This interaction that you had with the website is on the frontend. Everything that you see - the look and feel, the list of articles, the color scheme, and the buttons - all make up the frontend of the website. When you clicked the link a few things happen.

1. User clicks the link (Frontend): This action by the user is a frontend activity. The user sees the link on the page and clicks it. This will redirect the user to a new page on the website. This is like a customer seeing the menu in a restaurant and ordering a dish.

2. Sending the request (Frontend to Backend): At this point, the frontend sends a request to the server (backend), to get the contents of this page. This request is made in the form of an API call. (API is short for application programming interface. Think of it as a request asking the computer to do something. We will learn about how APIs work in another article.) As a user who requested the article, you don’t see this happening. You just wait for the page to load while the server is working to fulfil your request. So, the frontend has made an API call to the backend to access the contents of the article. Restaurant example: The request for the dish is sent to the chef in the kitchen.

3. Processing the request (Backend): Once the server receives the request it needs to process it. All the Learn Kraft’s articles data is stored in a database. So, the server will query the database to retrieve the contents of the article. All this is happening in the backend. Restaurant example: The chef prepares the dish. You can imagine what goes into making the dish.

4. Generating the response (Backend): Once the server has received the content of the article from the database, it will create a response. This response may include the article’s content and any other information that the frontend may have requested while making the API call. This includes things like image links, publishing date, etc. Restaurant example: The chef gets the dish ready to be sent to the customer.

5. Sending the response (Backend to Frontend): At this stage, the server will send the response to the frontend (user’s browser) where the user is waiting for the article’s content. Restaurant example: A waiter takes the dish from the kitchen to the customer's table.

6. Rendering the page (Frontend): Upon receiving the response, the frontend will render the contents of the article on the web page. While doing so, the frontend will also perform some other actions such as applying styles to the content, adding interactivity, and maybe some minor programming activities such as changing date formats, etc. Over time, frontend frameworks have become very powerful and can do a lot more than we can imagine. (We will learn about Frontend frameworks in another article.) Restaurant example: The dish is served to the customer.

7. User views the article (Frontend): Finally, the user views the article. As you can easily guess, it’s a frontend activity. Restaurant example: The customer enjoys the dish.

And there you have it! The next time you click on an article, just picture a little digital army scrambling behind the scenes, turning your casual click into your next intriguing read. It's like a secret behind-the-screen party every time you browse! Generally, all this takes just a few milliseconds, but depending on the quality of the server, the network, and a host of other factors, it may take more time.

Components of Frontend and Backend

By now, we have a better understanding of frontend and backend. The frontend is what the user interacts with directly. It is sometimes called the ‘client-side’. The backend, or server-side, is everything that happens behind the scenes to make that interaction possible. Let’s look at the important components of frontend and backend.

Frontend

HTML: HTML is the markup language that gives a webpage its structure. It provides the basic structure of the page, which is then enhanced and modified by CSS and JavaScript.

CSS: CSS is the presentation language that makes the page you created with HTML presentable. CSS is used for creating layouts, colors, fonts, and more—it makes everything look good.

JavaScript: This is the programming language that makes your website interactive, like adding animations, and creating interactive forms.

Frameworks and Libraries: These include tools like React, Angular, and Vue.js, which provide pre-written code to make it faster to develop and add interactivity to your website.

Responsive Design: This involves making sure the website looks good on any device, from smartphones to huge screens.

Backend

Server-side Programming Languages: These are programming languages like Python, Java, Ruby, PHP, and Node.js that you use to write the backend code. They are used to create the logic of websites, interact with databases, and more.

Databases: Databases are digital storage where all the website's information is kept, like a huge digital filing cabinet. They are essential for storing, retrieving, and manipulating data. Examples include MySQL, PostgreSQL, MongoDB, and Oracle.

Server: This is the powerful computer where your website or application lives. It is the server that serves, or sends, your website to users' browsers.

APIs: APIs are helper programs that allow different software applications to communicate with each other. In the case of a website, APIs are used to connect the frontend and backend of a web application, for sending requests and receiving responses. Think of APIs as waiters in a restaurant. You ask for data (order meal), the API (waiter) communicates your request to the system (kitchen), then brings back the data (your meal) for you to use (eat). APIs make different software talk to each other, just like a universal translator!

Middleware: A go-between software that helps different parts of the website work together smoothly. For example, a server will use middleware to retrieve information from a database.

Frameworks: Backend frameworks are helpers that can speed up development time by providing pre-written code for routine tasks. Some examples of backend frameworks are Django for Python, Express for Node.js, and Rails for Ruby.

All these components come together to make a website work.

Netflix and its Frontend and Backend

Now that we have a fair understanding of the concepts of Frontend and Backend, let’s take the example of Netflix, and see how it applies to this popular streaming device.

Frontend

Netflix has many frontend applications for various platforms. As their customer, you can access Netflix from a web browser, smart TVs, mobile devices and even gaming consoles.

Here is what happens in the frontend:

User Interface: The frontend is what you see when you open Netflix on your TV or mobile phone. You see the list of movies and TV series, your watchlist, upcoming shows, your browsing history, etc.

Interaction: This involves the viewers' interaction with the Netflix app. This may include browsing the catalog, playing a video, pausing, fast forwarding, adjusting settings, etc. On a TV, all this happens through your TV remote.

An interesting and unique frontend, or client-side, feature of Netflix is the adaptive bitrate stream. Using this tech, Netflix changes the quality of your video dynamically when it detects a change in your network speed. That's why, if you're watching Netflix and your internet suddenly slows down, instead of the video freezing or buffering, the picture quality only becomes a little fuzzy for a few seconds.

The frontend of Netflix is built using JavaScript, using a popular frontend framework called React.js that helps it build a dynamic and responsive user interface.

Backend

Netflix has a very powerful backend that handles millions of requests per day streaming video content to its users around the world. Here's what the backend does:

Content Delivery: Since Netflix has users around the world, how far a server is from its user has a direct impact on how much time it takes to serve a request (buffering). Netflix uses a Content Delivery Network or CDN to distribute its content. A CDN involves many servers placed strategically in different parts of the world so that content can be delivered quickly and with minimal buffering.

Personalization: One of the strong points of Netflix is personalization. As you watch content, Netflix personalizes your experience, recommending movies and series based on your watch history and preferences. All this is done by algorithms that run in the backend.

Authentication / Authorization: When you log in, the backend validates your credentials and determines what you have access to based on your subscription.

Payment Processing: The backend also handles the billing and payments for user accounts.

It's worth noting that Netflix runs its infrastructure on the cloud and is a heavy user of Amazon Web Services (AWS). Most of their infrastructure runs in the AWS cloud.

With this example, we’ll conclude this article. I hope you now have a better understanding of the backend and frontend of software. Until next time!

Imagine you're a photographer who has captured breathtaking moments throughout your travels. Your camera has thousands of high-resolution photos, each representing a unique memory. However, as you accumulate more and more images, you begin to face a problem - where do you store them all?

This scenario highlights a familiar dilemma individuals and businesses face in the digital age. With the exponential growth of data and the need for secure and accessible storage, traditional methods are often insufficient. This is where cloud computing comes to the rescue.

A few years back you might have transferred all these images to your computer or a backup drive, but in 2023, the chances that you save them on a cloud service such as Dropbox, Google Drive, or OneDrive are higher. But have you ever thought about how these services work, where they store the data, and how they make it accessible to you over the Internet?

If you think about it, the foundation remains the same. It is still a computer located somewhere else instead of your home. We call this computer a server in the cloud. The computer has the storage where your photographs are stored. In addition, this computer is connected to the internet and runs special software that makes it possible for you to store and retrieve your pictures from it. That's basically what we mean when we talk about cloud computing.

In cloud computing, the term cloud is used metaphorically to represent the idea that the exact location of the computer and the data storage is abstracted and hidden from the end-user. The end users don’t have to worry about the physical location of the servers. Instead, they just focus on what they need to do on the server (In our example, manage photographs).

Let’s understand how cloud computing fits into the business scenario.

On-premise - Traditional way of deploying software

Traditionally, software companies installed their software on the buyers' own servers, which are typically located on the buyers' premises. This arrangement is known as an on-premise deployment model. The servers would be accessible to the company's employees over their own network. This approach has been the standard practice for software delivery for many years.

However, this method has many disadvantages:

• You have to make a huge upfront investment in buying these servers, storage, and networking. This is what we call Capital Expenditure. Depending on the type of software business, this cost could run into millions, and was one of the biggest hurdles in getting into a software business (It was literally an expensive business!).

• Scaling is really difficult. If the demand is growing, you would need to physically add more computers or replace the existing ones with bigger computers to meet the demand. This is costly and time-consuming and sounds like a hassle.

• You have to maintain this infrastructure, which means you need to invest in space, power, and air conditioning and also have dedicated IT resources and experts that maintain this infrastructure and software.

There are also other disadvantages such as managing downtime, security and backups.

On the cloud - The new way

While the idea of the cloud has been around for quite some time, things really changed when in 2006, Amazon Web Services (AWS) was launched and they came up with their first service called Elastic Compute Cloud (EC2). You can think of EC2 as really large computers accessible over the internet, which you could access by paying a fee to AWS. These large computers, instead of calling them servers, AWS called them instances. Once you had access to an instance, you could do anything on it, just like you would do on your own home computer or an on-premise server. AWS EC2 was the first cloud computing service that was made available to the public.

The idea is very simple. Instead of buying your own servers, you could just rent the servers on AWS, run your software on it and access it over the internet.

This marked a revolutionary change.

• Rent vs. buy: Instead of buying, you could rent servers at a small fee. Your CapEx has turned into OpEx.

• Easy to switch providers: Since you are paying monthly (or sometimes yearly), it’s easy for you to switch providers. Currently, the four largest players are Amazon AWS, Google Cloud, Microsoft Azure, and Alibaba Cloud. Apart from this, there are plenty of small and vertical players. If you are not happy with one cloud provider, you are not tied to them for life. You can easily move from one provider to another in a matter of weeks.

• No maintenance: All the maintenance and upgrades of the cloud are done by the cloud provider, so you don’t have to do it.

• More secure: When you maintain your own on-premise infrastructure, you are responsible for security and data protection. However, in the case of cloud computing, the cloud providers are responsible for it. Cloud providers dedicate significant resources on security, and the general industry perception is that a cloud server is much more secure than any on-prem server.

How is the Cloud Organized?

As we mentioned, a server is just a large computer. However, it doesn’t look like a PC or a laptop. It doesn’t need many things that a typical computer has such as a screen or a keyboard. Servers have multiple CPUs, multiple hard drives and large amounts of RAM. Servers are designed to be space efficient. There are many types of servers such as rack-mountable servers, blade servers, and tower servers. Servers typically look like below:

The physical place where servers are stored is called a data centre. A data center will have many racks, and each rack will hold many servers typically stacked over one other. This will include, servers, networking equipment, storage devices, and other equipment needed to connect and power the servers.

The racks with servers in a data center look something like this:

Types of Cloud

There are three main models for cloud computing services: Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). Let’s look at each of them in detail:

• Infrastructure as a Service (IaaS): This is the most basic form of a cloud where you get the very basic elements a computer should have. This is like renting a computer. You get the necessary hardware over the internet, but you're responsible for the software. This includes the operating system, applications, and other components. You can customize and control everything, but you also have to manage everything yourself. Some examples of IaaS are Amazon's EC2, Google Compute Engine, And Microsoft Azure’s virtual machines. Since you are getting the most basic infrastructure, IaaS is the cheapest cloud offering.

• Platform as a Service (PaaS): In PaaS, a few things are already installed and configured for you. It’s like a pre-set computer that comes with a few software such as the operating system and a few other software pre-installed. You don't have to worry about the operating system or lower-level infrastructure. This makes it easier for you to start and you can just focus on developing, running, and managing your application. This is good for developers who want to focus on their code without worrying about the underlying system. Google App Engine is an example of PaaS. Some other examples are Heroku, Microsoft Azure App Service, AWS Elastic Beanstalk, Amazon RDS, and GCP Big Query. We will learn about what these services exactly do in another article.

• Software as a Service (SaaS): SaaS has become the most popular software distribution model in the world. This is when a vendor builds fully-featured software and makes it available over the cloud to its end users. You don't have to worry about the underlying hardware or software at all. You just use the application, and the provider takes care of everything else. This is the most user-friendly type of cloud service, and it's what most people think of when they hear "cloud computing". Examples include Google's Gmail, Microsoft's Office 365, Twitter, Salesforce, Zoom, Slack, Dropbox and many other software that you use every day.

How Companies Deploy Cloud?

Earlier we discussed cloud servers provided by large players such as Amazon AWS, Google Cloud Platform (GCP), and Microsoft Azure. All these cloud servers are available for anyone to purchase and are generally referred to as public cloud. This is like rental properties that are available for anyone to book and use.

Instead of a public cloud, a company sometimes may choose to build its own cloud where this cloud is exclusively available for this company. This is called a private cloud. It's like owning your own home. A company may have their private cloud located in its own data center, or it can be hosted by a third-party service provider.

Sometimes companies may want to deploy a combination of public and private clouds. It's like having a house and also renting a vacation home. This could be for security reasons where they will use a private cloud for tasks that need to be more secure and a public cloud for the rest of the tasks. Such an arrangement is called a hybrid cloud.

Sometimes companies may go for a multi-cloud model where they will use multiple cloud services from different cloud providers. This helps them avoid vendor lock-in, increase reliability, and take advantage of the best features from each cloud provider. As an example, a company may use Amazon EC2 for their compute infrastructure (basic hardware), they may use Google Cloud Platform (GCP) for Big Data Analysis, Amazon RDS or Azure's Cosmos DB for hosting their database, and use IBM Watson for their Artificial Intelligence and Machine Learning needs. This way they get the best of each cloud provider.

Thanks for making it to the end of this article! We've covered the key concepts and terms of cloud computing you should know. Hopefully, this piece has shed some light on the subject, making it a bit less mysterious. So, next time cloud computing pops up in a meeting or a chat with clients, you'll be ready to dive right in. Until next time, keep learning!