Automation can reduce company, industry and technology research times by up to 50 percent

Jul 19, 2018


Knowledge workers spend a lot of time searching for and preparing data. We experienced this first hand in years of innovation research. Automation is replacing much of that knowledge work. And that is good news. Automated research analysts like ours can significantly reduce company, industry and technology research times.

Unsurprisingly, "knowledge workers" spend a large part of their day searching for and preparing data. Estimates vary depending on the type of work they do. Data scientists, for example, spend up to 60 percent of their time cleaning and organizing data, and about 20 percent of their time collecting data sets. More generally, workers in the digital workplace, on average, spend 36 percent of their day looking for and consolidating information. Interestingly, 44 percent of the time, they cannot find the information the look for! Of course, in today’s fast paced world, time is money. The more time these workers spend doing basic or repetitive tasks, the less time they have for more value added activities.

We know about this very well. In more than 15 years of innovation research experience, I personally worked with diverse data sources (e.g. open databases, company websites) and collection methods (e.g. surveys, web scraping) and did a whole range of types of data analysis. The outputs of my research have ranged from simple one-pagers to scholarly journal articles, consulting and policy reports, and a book as well. Collecting data and cleaning and preparing datasets consumes between 30 and 50 percent of the time of these projects. Doing even the most basic analysis can take another 20 or 30 percent of the project time. That leaves very little time to produce quality outputs. And deadlines are deadlines. Furthermore, when the goal is producing, for example, a $5,000 industry report that could take one or two months from idea to final delivery, data and basic analysis work add up to thousands of dollars, an amount that clean data and reliable insights can help save.

This math applies to every new project one starts from scratch, regardless of whether one is a junior analyst or a seasoned researcher. The result is lots of repetitive tasks, new opportunities for mistakes, and a big waste of time and money in the form of unexploited similar work done in the past. This affects not only the independent researcher but also the global community of research analysts conducting similar research. Think about how much time could be saved, and how much more quality research could be produced, if at least part of those repetitive tasks and analyses were done just once, reliably, and their results made available to everyone.

The fact is that, as many other positions, analysts and researchers are slowly being replaced by software that takes over that kind of repetitive work. There are already some examples in Wall Street, and this will also occur in the more general research industry (although some evidence shows a still slow adoption process of artificial intelligence in the workplace). This is not the end of the Research Analyst position. We think this is actually an opportunity for analysts to focus on more value added research and let automated research solutions such as InnovationPulse do the heavy lifting in data preparation and at least part of the analysis.

We believe there are four basic requirements for an automated research solution and the insights it provides:

  • Quality in innovation research is very important, particularly when its aim is to support strategic decision-making. Insights can be more or less accurate, which is not a problem per se if information on the level of accuracy is also provided.
  • Transparency is important as well. "Black box" solutions may do a "good job", but it is impossible to know how good it is because of the lack of transparency.
  • Coverage is another valuable trait in more complete solutions that also include data gathering. Analysts should get the real picture when researching an industry sector, not only stories about the latest startup or that well known large company that all the mainstream media talks about.
  • Access. Last but not least, analysts need solutions that are affordable and accessible to everyone.

We keep those requirements in mind when we develop our automated research analyst [1]. The core of this analyst is a research engine that collects open data (such as patent data) and analyzes it in real time. Online dashboards offer insights in the form of metrics, tables and visualizations, which can be accessed from any computer with an Internet connection. Moreover, users can also contribute data and collaborate with fellow analysts and researchers to augment open data and improve the quality of insights for everyone. We strive to make this solution accessible to everyone, regardless of what country they work from, and help level the innovation playing field.

Our estimate is that this kind of tools can reduce company, industry and technology research times by up to 50 percent. They can also help discover unexploited good talent, new business opportunities, and valuable technologies that remain generally unknown because of a lack of better innovation insights. Every researcher, every analyst should take advantage of this new technology to improve and add value to their work. Solutions like ours do not require you to bring any data or install any software – they are ready to use. Moreover, they include support as well. You just need to bring your research ideas. More at


[1] The current open beta version is fully functional but limited in the number of data sources and features we would like to have. We plan to add many new features to make it more complete, accurate and insightful. We will also continue working to make it even easier to use.

How to cite

InnovationPulse (Jul 19, 2018). Automation can reduce company, industry and technology research times by up to 50 percent. Retrieved from

About this post

The purpose of this blog post is to illustrate the kind of insights users can obtain by using our web app. None of the insights or information we provide are intended as investment, tax, accounting or legal advice.