Business success depends not only on technical implementation but also on careful analytical work and planning. To avoid losing time and money, it is better to study the market at once: analyze demand, competitors, target audience and external factors so that you can make decisions based on real market opportunities rather than on your hunches.
Qualitative research opens new trading opportunities, helps to reveal the competitors' advantages, and explores the target audience's desires. Companies that analyze the market reach their expected revenue levels faster.
What is data-driven marketing?
Data-driven marketing is marketing based on analyzing large amounts of information about all business processes, mostly about consumers. Now marketers spend over $6 billion annually to create solutions using data management platforms and demand-side platforms to get their message out to users. More than 60 percent of companies are actively using big data to:
Data-driven principles are simple: you make decisions based on an analysis of numbers. Intuition and personal experience take second place. Specialists must be able to interpret data and make hypotheses. In addition, you should take care of how to mine, store and visualize those numbers. This will require both meters and analytics services, as well as technology: machine learning, predictive analysis and artificial intelligence. Classic data-driven marketing helps:
By collecting statistics over periods, which you need to systematize somehow, you will find correlations and create models of audience behavior, predict sales and set KPIs.
How do companies gather data?
Companies collect data in different ways from almost every corner of the internet. Some methods are highly technical, while others are more deductive. Information can either be collected or bought.
Some data vendors offer already compiled compilations, but they may not contain the values you need. And you won't have the ability to collect or augment them.
Data collected by yourself or with the help of a third party is more complete and always up-to-date. You'll be able to aggregate it independently according to your needs and business requests and work with it as you want. Companies use different tools to collect data:
Those companies that maximize data collection and know how to work with it run almost all of their operations: marketing, sales, staffing, updates and enhancements, deliveries. The more data they collect, the more information they will have about their users, and more importantly, they understand the context of what they do.
What is web scraping?
Web scraping is an automated process of extracting data from certain site pages according to certain rules using bots. A bot is a program in any programming language whose logic of operation is aimed at retrieving data from websites.
The bot's task is to retrieve web page HTML code that contains information it is interested in. After receiving the code, scraper analyzes it, extracting all necessary data. This way, the bot can open all possible site pages and get information.
Web scraping can be used to gather any information useful for businesses, whether it's data about users, products, competitors, and so on. Everything we talked about in data-driven marketing and will talk about below can be obtained through scraping.
Web scraping: is it legal?
The ultimate question is about how you plan to use the extracted data. In many countries, it is legal to collect data for public consumption and use it for analysis, for example. But it is always illegal to collect confidential information without permission for sale or to use the material as your own without identifying the source.
Websites have their own "Terms of Use" and copyright information, which can be found on the homepage that state how their data can be used and how you can access their site. It's a good idea to check through the website's terms of use before beginning any scraping project.
Scraping also falls under several legal provisions. Some of them are Violation of the Computer Fraud and Abuse Act (CFAA), Violation of the Digital Millennium Copyright Act (DMCA), Copyright Infringement.
Why conduct a market analysis?
Market analysis is the collection and processing of information about the environment in which an enterprise will operate or is already operating. Usually aimed at reducing the risks of entrepreneurial activity. The results obtained allow making management decisions, which are always accompanied by risk in conditions of constant market changes and uncertainty in the behavior of both consumers and competitors. It should be conducted when you:
The final work results and company stability depend on this information. Market analysis is needed for:
How to conduct a market assessment
The set of trackable metrics is formed directly from the objectives. Focus on key parameters and collect data on factors that accurately or at least hypothetically influence those metrics.
Don't collect data just in case. Data-driven marketing should help you optimize your business, not waste all of your resources on collecting useless information.
General market activities study
To clearly define a segment and niche for a detailed analysis of the commodity market, it requires data on market capacity, dynamics, and changes.
Market capacity is the market size of a particular product or service, expressed in the total volume of product sales over a given calculation period. It is usually calculated for a specific territory and a certain period and depends on:
Based on the capacity indicator you can estimate the prospects for sales growth and the maximum income that can be obtained in the selected segment.
To determine the dynamics and prospects of the market, take the capacity figures for the last 5 years and determine whether there is an increase in demand for goods? If demand is increasing, then the dynamics are positive; if it is decreasing, then the dynamics are negative. If the indicator is negative, find out the reasons, assess the situation and other proposals in the market, and then make a decision - is it worth entering the market or not.
Demand and target audience analysis
The next step is to determine the category of customers who bring most of the company's income. Consumers are divided into groups based on general characteristics: age, gender, social status, income level and others. In the course of such work, we consider not only consumer inclinations and customs, habits and preferences, but also clarify the reasons for the behavior of specific consumer groups. This makes it possible to anticipate the future structure of their interests.
The demand for the product is determined by:
Demand for products will show how much customers are interested in the product at a particular time or are in the mood to buy. To analyze demand as accurately as possible, determine market geography and target audience. Look for data to estimate the market in:
How to analyze demand
Demand analysis for a product or service will help you understand how much demand there is for the offer and track the dynamics of users' interests. Collect information before analyzing. It can be obtained through:
Surveys
A survey is a simple and common way to collect data that can be conducted online. Answering the questions will help determine customer expectations and reactions to the product launch, how people will position the new product, what they would like to change in existing products or expect to see in new ones and trace market-shaping patterns.
Questions are created according to the needs of the business using tools such as Google Forms, Oprosso, Examinare. Or ready-made templates from the Internet are used. These can be found on Survio, Typeform, or QuestionPro.
Advertising research
A test advertising campaign is a good starting point for testing product demand. You can place ads in Google or Bing to evaluate the interest of buyers, the value of the offer, the effectiveness of the UTP, the cost of the targeted action, the approximate amount of traffic, the size of the advertising budget, and its payback.
With Facebook Ads Library you can check the contextual advertising of competitors, what is being used in the ads to attract customers and find out how many users of social networks correspond to your audience.
Google Keyword Planner is a tool designed to collect and forecast keyword query statistics and create new advertising campaigns. The service displays average number of search queries by keyword and close variants per month; analyzes word statistics; gets forecasts for existing campaigns; shows a list of words that competitors use for advertising.
Competitor research
If you are launching a new and unique product, you shouldn’t be afraid of competition in your segment. In other cases, you need to study the advantages and disadvantages of competitors in advance.
Study both direct and indirect competitors, they are all a valuable source of information. Pay attention to those who work in the market for a long time and even in a crisis keep the bar and those who lag, to take into account their mistakes. All the information gathered will show who you will have to deal with when you enter the market. Be sure to look at:
For a more detailed analysis of competitors, tools and services such as SimilarWeb, SEMRush, Spywords, Screaming Frog, Marketing Grader, and others are used.
Trend analysis
Analyzing trends will give you an understanding of how your business is performing and predict where current business practices will lead. They can provide information on what's best to do and what's not. For example, this applies to investing.
You can use trend analysis to improve your business by identifying successful areas, tweaking the development of lagging areas, and providing evidence for decision-making.
To find out what's trending, use Google Trends or Twitter Trend Takeover.
Google Trends
Google Trends is used to assess general trends in demand dynamics on Google. Such information is especially useful for large stores or when starting to sell new products. The service allows you to select a region, set a period, and look at the situation in searches for images, news, products, and YouTube. Google Trends has a feature that compares trends for related queries.
Twitter trend takeover
Trends on Twitter appear in or near each user's feed and display the latest popular topics that you can use for market research and trending.
Also, Facebook used to have a "Trending" section to keep users updated. Current topics showed headlines that people were talking about all over Facebook. Unfortunately, Facebook has removed this feature.
Data visualization
It is difficult to conclude the results of a study if the information is not organized. To make the analysis more efficient and convenient, structure the data; otherwise, all the information collected will be hard to comprehend. Present the information in graphs, charts, tables, histograms, or maps. Usually, analysis tools visualize the data at the same time. You can also use special services, such as Google Charts, Tableau, Qlik, Infogram, Orange, Power BI.
When analyzing and visualizing market trends, it is best to combine data from Google Trends and Keyword Planner with data from website scraping for a more complete picture of what is happening. For the analysis, data from 500M products from more than 1.1M stores on the Shopify platform was gathered for November 2021.
We'll put the number of search queries from Google Keyword Planner and the number of products added to stores into relative figures, like Google Trends, where 100 is peak popularity and 0 is no data.
Example: Vitamin C serum
Vitamin C serum is currently one of the top products in demand, according to online searches. Interest in the product has gradually grown and has remained steadily popular for several years, which is also shown in the chart below. It’s coupled with a growing interest in skin health.
Image source: internal tool
Example: Massage guns
Body massage guns have become very popular on the market of massagers, which has attracted the stores' attention. In the comparative graph, you can see the positive dynamics of demand for the product and how soon online stores began to add automatic massagers to their assortment. The peak of interest came in Q1 2021 and is still not subsiding.
Image source: internal tool
Example: Overshirts
Recently, there has been an increase in requests for oversize shirts, designed to be worn over other things like a jacket. That is why they are called overshirts, they are versatile. Over time, interest in the product is only increasing, as can be seen in the chart. The peak of interest came in Q3 2021, when overshirts became a fashion trend.
Image source: internal tool
Example: Yoga mats
In times of pandemics, the situation has made people enthusiastic about yoga and other fitness routines. The yoga mat is a must-have for people working out in fitness clubs as well as those who do it at home.
People's interest peaked in Q3 2020, when there was a new wave of disease and people began to stay home again. Unfortunately, after that interest in yoga mats died down. By Q3 2021, interest has slowly started to grow again.
Image source: internal tool
Closing thoughts
It is unthinkable today for a company to exist without knowing the basic drivers of the market mechanism. Marketing analysis serves as the basis for developing a whole range of measures, including adapting goods for new target segments or markets, adjusting prices depending on competitors' ones, and sales promotion. Competent market research significantly increases the awareness of management, reduces entrepreneurial risks, and increases the validity of management decisions.
Sergey Ermakovich is the chief marketing officer at Techvice.
Welcome to the VentureBeat community!
Our guest posting program is where technical experts share insights and provide neutral, non-vested deep dives on AI, data infrastructure, cybersecurity and other cutting-edge technologies shaping the future of enterprise.
Read more from our guest post program — and check out our guidelines if you’re interested in contributing an article of your own!