The basis of Azure Data Factory

In the world of big data, raw, unorganized data is often stored in relational, non-relational, and other storage systems. However, on its own, raw data doesn’t have the proper context or meaning to provide meaningful insights to analysts, data scientists, or business decision makers. Big data requires service that can orchestrate and operationalize processes to … Continue reading The basis of Azure Data Factory

Introduction to Temporal Windows (Azure Stream Analytics)

In applications that process real-time events, it is common to perform some set-based computation (aggregation) or other operations over subsets of events that fall within some period of time. Because the concept of time is a fundamental necessity to complex event-processing systems, it’s important to have a simple way to work with the time component … Continue reading Introduction to Temporal Windows (Azure Stream Analytics)

Microsoft Azure Data Lake Store: An Introduction

The Azure Data Lake Store service provides a platform for organizations to park - process and analyse - vast volumes of data in any format.  With increasing volumes of data to manage, enterprises are looking for appropriate infrastructure models to help them apply analytics to their big data, or simply to store them for undetermined … Continue reading Microsoft Azure Data Lake Store: An Introduction

Sharding pattern in Azure

Divide a data store into a set of horizontal partitions or shards. This can improve scalability when storing and accessing large volumes of data. Distribution Models Once the hardware resources, server nodes, for deploying a distributed database are available, a distribution model should be chosen to leverage the cluster capacity. Roughly, there are two paths … Continue reading Sharding pattern in Azure