As university courses kickstart again, concerns are being raised about the noticeable lack of insights into web intelligence in the curricula – an absence that could hinder the growth of data-driven technologies. With perspectives that the alternative data industry could be worth £120bn by 2030, the importance of educational insight into web intelligence for students has never been greater.

Juras Jurnas, COO at Oxylabs, stresses the urgency with which universities need to educate students on the value of web data scraping, particularly in relation to the study of artificial intelligence (AI) and machine learning. Jurnas argued, "To ensure that data-driven technologies and industries reach their full potential, web intelligence needs to get the right exposure and education, and to achieve this, it must be taught at universities."

He also addressed the relevance of web scraping in AI and machine learning studies, which are gaining popularity in higher education. "AI and ML studies are becoming very popular, with top universities offering dedicated courses. Students in these programs often lack the proper datasets to develop and train ML algorithms, and web scraping knowledge would help them build quality datasets for more efficient work."

In an attempt to bridge the knowledge gap and aid academics with big data gathering using web intelligence solutions, Oxylabs launched a charitable initiative called 'Project 4'. This initiative allows universities and NGOs free access to data scraping tools and supports research on data that addresses meaningful social missions.

Highlighting the potential that alternative web data analysis holds for social, economic and psychological studies, Jurnas noted, "For example, investigative journalists and political scientists use alternative data to study a wide range of issues, from tracking the influence of lobbyists by investigating visitor logs from government buildings to monitoring prohibited political ads and extremist groups in public social media platforms and forums."

The Oxylabs COO also addressed the legal worries surrounding web intelligence. "Web intelligence is a new industry, surrounded by various legal concerns, and it still lacks clear regulation. This is discouraging some researchers from leveraging public web data in their studies. There is nothing inherently illegal about web scraping if data is gathered ethically – it automates activities that people would otherwise do manually."

Oxylabs, together with other prominent web intelligence companies, recently introduced the Ethical Web Data Collection Initiative - a commitment to promoting ethical data-gathering practices and industry-wide standards. Jurnas concluded, "Web scraping has yet to gain traction in the public eye and academia. However, with the sheer volume of web data increasing exponentially, web intelligence will slowly become an inevitable part of scientific research."

Oxylabs established its reputation in 2015 as a top-tier proxy and public web data acquisition solution provider, enabling businesses of any size to harness the power of big data. In 2022, Oxylabs was recognised as the fastest-growing public data gathering solutions company in Europe according to the Financial Times' FT 1000 list.