What is data, and how does “big date” work

Possibilities and principles of data processing using Big Data technologies

In general, these are exact numbers, defined quantities. In the environment, the data indicate the abundance of precipitation and the degree of humidity. In the field of trade — the average check and the lifetime value of the client. In the marketing field — age and gender. In customer service management, there is an index of customer loyalty and customer satisfaction.

The data is real and measurable.

Assumptions are individual and cannot be measured.

Now, having found out what data is, let’s deal with the concept of “big date”

What is big data?

Surely you have already translated the phrase “big data” as “a lot of information”. But over time, this term has acquired an even broader meaning.

Big data is a set of tools, methods and approaches to data processing. First of all, they are used in order to apply the conclusions obtained for solving certain tasks.

Let’s say forecasters collect historical weather data and modern meteorological indicators → process them → make a weather forecast in their locality for several weeks ahead. Thus, big data has an impact on solving problems in almost every industry.

Big data is a combination of both ordered and unordered data:

• ordered: schemes, lists, everything where organized and ready-to-use information is provided;
• disordered: satellite photos, surveillance cameras, videos, audio recordings of conversations, etc. With their help, you can determine the data, but you will not be able to fix them in the table. To process disordered data, a recognition function is installed in the system, artificial intelligence and machine learning are possible.

Big date has the most important characteristics — the so-called VVV, formulated in 2001 by Meta Group:

• Volume — the volume and quantity of all data;
• Variety – variability, that is, a variety of types of data;
• Velocity — the speed of data growth and processing.

To date, there is a tendency to decline to 4V (confidence is added), for example, as to IBM Big Data Analytics. In exceptional cases — even to 6V (plus durability and value of information).

How do they arise?

It is we who are engaged in the production of a huge amount of data: wanting to communicate on social networks, visiting the website and giving an agreement on the use of cookies, using GPS, creating orders in online stores. This represents a digital footprint.

Some data is created by machines: while connecting a smartphone to Wi-Fi in a public place, triggering alarm sensors, fixing the coefficient of operation by the equipment of the enterprise. This data is accumulated daily and every second from all over the world, which allows you to record everything that happens.

Big Data Capabilities

Big Date provides assistance in global things:

• Discovery of new planets. NASA, analyzing the data, creates models of future flights into space.
• Predicting emergency situations. Scientists in the field of genealogy and volcanology, processing a large amount of data, can predict possible cataclysms and create effective behavioral models for victims to protect them from danger.
• Investigation of incidents. Jawbone calculated the central point of the earthquake in California after studying user data from thousands of fitness trackers that record the moment of awakening. The logic is simple: people who woke up earlier than everyone else are closer to the earthquake source. And this helped to determine which point was the beginning of the earthquake. This story is an example of an original and masterful application of big data.

Big Data is also used to solve smaller scale tasks:

• Chevrolet uses a large amount of data to improve the models of its cars. To do this, special sensors are installed in cars that accumulate information about the oil condition, movement speed, tire pressure, etc.
• Platforms for fans of movies and TV series help to make a choice of what to watch, based on user preferences and viewing history.
• Dating resources are engaged in bringing people together using a volume of data.

Big data also has an impact in narrower business tasks:

• In 2014, Sberbank resorted to the use of a photo analysis system for the purpose of identifying bank customers, which is based on a biometric platform. As a result, fraud was reduced by 10 times.
• VTB-24 enters into cooperation with analytical databases (Teradata, SAS Visual Analytics and SAS Marketing Optimizer) for segmentation and management of customer loss based on big data; creation of financial reports, analysis of business responses.
• Google.Trends predicts seasonal demand activity, also using big data.

Principles of operation

The main essence of the work of big data is that the more knowledge there is, the more accurately you can create a forecast for the future. If you know that for several hundred years in your locality rains began on September 20 (if it was 25-27 °C in summer) or October 5 (if it was 28-30 °C in summer), then it is quite realistic to predict precipitation this year. Maybe this is an unfortunate example, but that’s about how big data works.

The juxtaposition of data and their relationships allows us to find previously unknown patterns. And thus, it allows you to detect the big picture and understand how the system functions in the aggregate, how to manage it and how to predict its further behavior.

To this end, based on the available data, various cases/techniques/options are modeled (depending on the industry) and imitation is carried out. What will happen when such conditions are set? And if the temperature rises by a degree? And if the humidity becomes 25% lower? The system develops a model of the possible development of situations and demonstrates how a change in each specific parameter affects the final (probable) result.

Of course, the process is automatic, taking into account that terabytes of data are used and hundreds of thousands of possibilities are simulated.

How is the data processed?

The data itself does not make sense to a person. Our brain cannot draw conclusions from the scheme of sales by thousands of points, not to mention terabytes of various data that may not be ordered yet (like photos or videos).

In order for the data to become usable, they need to be processed — they analyze and get results that a person can use.

The main techniques and methods of data analysis:

• identification of images;
• predictive analytics;
• multidimensional analysis;
• analysis of statistics;
• Data Mining;
• crowdsourcing (the inclusion of human capabilities to find a common solution to specific problems or the implementation of projects);
• machine learning;
• INS (artificial neural networks);
• Simulation simulation;
• visualization of analytics data, which was already discussed above.

For all this, the use of special technologies is implied:

• NoSQL;
• MapReduce;
• Hadoop;
• R;
• Hardware solutions.

It is likely that you have heard some of these names somewhere.

For whom are they needed?

In some cases, Big data is presented as an integral component for business.

However, this is far from the case.

Big date is not a medicine, but a tool that tends to develop. If it makes sense for companies like PrivatBank or Rosetta to implement big data, then local small/medium-sized businesses most often do not need it.

First of all, with a small amount of work, there will be none of the VVVS.

Also, the introduction of technologies of this kind for small businesses is usually inappropriate.

Thus, today big data is an advantage of large businesses and research institutions. However, remembering the Jawbone technique using data from fitness trackers… Original thinking may well help big data and small businesses.

What are the reasons for the popularity of big data?

First of all, it is necessary to highlight such.

Big data is useful for customers

They make advertising suitable, in accordance with the preferences of users, it does not irritate and presents useful products. Choosing in an online store becomes much easier and more comfortable: after the purchase, you do not see an advertisement for the same product (why do you need another tablet if you have already purchased one recently?).

But the advertising of the accompanying new headphones, films, tablet cases and other accessories is displayed. It is practical and useful. This does not irritate the user. And the customer is calm, because his company does not lose anything.

Big data can transform advertising from “sniffing” into “effective advice”.

Have an impact on competition

Previously, brand marketing specialists, while working with a potential audience, were based on “small data”: gender, age, location. To date, all companies in various industries have this data. It is becoming more and more difficult to build up from competitive companies every year, the importunity of advertising has increased significantly.

Little Data	Big Data
Guys 20-25 years old	Guys 20-25 years old who study at bachelor’s and master’s degrees, have a dog at home, smoke
Women from Odessa	Unmarried residents of Odessa, live in a rented apartment, eat in fast food establishments

Big data allows you to identify a potential client with maximum accuracy, target him by a large number of parameters: lifestyle, visiting places, philosophy of life. Business addresses its customers more accurately, and competitiveness is able to effectively develop the market.

Thus, you can identify your audience much more accurately, pay attention to the most effective sectors, and you will be aware of where and how to reach them!

They help to make an assessment of advertising

The use of big date allows you to evaluate ongoing campaigns as a whole, carefully amend them without suspending advertising activity and specifically know the effectiveness of all advertising messages. And this means lower costs, high efficiency, increased return on investment, profit-profit.

Why did they talk about Big Data only recently?

Many factors are known to this. The history of the emergence of big data (at that time an unknown concept) and its development to the current state is very rich and is largely based on the reality of the market, the development of technology, the achievements of some evangelical companies of those times.

In fact, at the heart of everything is just one fact: people have learned how to collect and process big data.

Before the “Big Data Era”	In the “Big Data Era”
Advertising of children’s clothing was shown to all girls over the age of 18	Campaigns also target information from the history of search queries: if a woman is interested in “how to take care of a baby” or “what name to give her child”, it is clear that she is a specific part of the target audience.
The car often broke down and had to be taken to a car repair shop for a manual inspection of all the details according to the “symptoms”	Electronic sensors are engaged in recording all the necessary indicators in the car and give a signal if at least some process deviates from the normal mode
For people who are sick, doctors put thermometers in order to then record the body temperature in the patient’s medical record	Dozens of sensors are attached to the patient’s body, which are able to independently measure body temperature, blood pressure and pulse; carry out an additional measurement in a given period of time and monitor the dynamics of the condition

In addition, they have not only learned how to collect data, but also how to process it. Hundreds of terabytes of information need enormous computer power to process. Several decades ago, these capacities were not publicly available. In the modern world, they are getting cheaper every year

For example, according to Mkomo, the price of storing 1 GB of data has become lower by more than 400 times since the beginning of the XXI century.

This makes it possible for enterprises to implement big data for everyday work.

What’s next?

Now you know a lot of ways in which big data is used in the world. If you consider it appropriate to introduce them into the activities of your company, then it makes sense to delve into this topic further. If you are still far from big data, then you are now well aware of the market and technological reality. It’s not for nothing that they say that information is the greatest power in the world.