Tomorrow's workplace will be run on mind-boggling amounts of data. To make sense of it all, businesses, developers and individuals will need better artificial intelligence (AI) systems, better trained AI workers and more efficient number-crunching servers.

While big tech companies have the resources and expertise to meet these demands, they remain beyond the reach of most small and medium-sized enterprises and individuals. To respond to this need, a Concordia-led international team of researchers has developed a new framework to make complex AI tasks more accessible and transparent to users.

The framework, described in an article published in the journal Information Sciences, specializes in providing solutions to deep reinforcement learning (DRL) requests. DRL is a subset of machine learning that combines deep learning, which uses layered neural networks to find patterns in huge data sets, and reinforcement learning, in which an agent learns how to make decisions by interacting with its environment based on a reward/penalty system.

DRL is used in industries as diverse as gaming, robotics, health care and finance.

The framework pairs developers, companies and individuals that have specific but out-of-reach AI needs with service providers who have the resources, expertise and models they require. The service is crowdsourced, built on a blockchain and uses a smart contract—a contract with a pre-defined set of conditions built into the code—to match the users with the appropriate service provider.

"Crowdsourcing the process of training and designing DRL makes the process more transparent and more accessible," says Ahmed Alagha, a Ph.D. candidate at the Gina Cody School of Engineering and Computer Science and the paper's lead author.

"With this framework, anyone can sign up and build a history and profile. Based on their expertise, training and ratings, they can be allocated tasks that users are requesting."

Democratizing DRL

According to his co-author and thesis supervisor Jamal Bentahar, a professor at the Concordia Institute for Information Systems Engineering, this service opens the potential offered by DRL to a much wider population than was previously available.

"To train a DRL model, you need computational resources that are not available to everyone. You also need expertise. This framework offers both," he says.

The researchers believe that their system's design will reduce costs and risk by distributing computation efforts via the blockchain. The potentially catastrophic consequences of a server crash or malicious attack are mitigated by having dozens or hundreds of other machines working on the same problem.

"If a centralized server fails, the whole platform goes down," Alagha explains. "Blockchain gives you distribution and transparency. Everything is logged on it, so it is very difficult to tamper with."

The difficult and costly process of training a model to work properly can be shortened by having an existing model available that only requires some relatively minor adjustments to fit a user's particular needs.

"For instance, suppose a large city develops a model that can automate traffic light sequences to optimize traffic flow and minimize accidents. Smaller cities or towns may not have the resources to develop one on their own, but they can use the one the big city developed and adapt it for their own circumstances."

More information: Hadi Otrok et al, Blockchain-based crowdsourced deep reinforcement learning as a service, Information Sciences (2024) DOI: 10.1016/j.ins.2024.121107