info@embecosm.com

There is a common misconception that modern AI needs a large data center and lots of GPUs to run. This may be true when training large models, and it is true that modern Large Language Models (LLMs) are quite compute intensive.

However there are a great many very useful AI models which can run inference on much smaller devices. Examples include image and audio processing, object detection and classification. Over the past year, Embecosm has worked closely with Mosaic SoC of Zürich developing an implementation of ExecuTorch for a small, low power, 32-bit RISC-V based multicore microcontroller (using the CV32E40Pv2 from the Open Hardware Foundation).

ExecuTorch is a sub-project of the Linux Foundation’s PyTorch project, which provides support for AI model inference on resource constrained machines. PyTorch was originally created by Meta and is one of the most widely used frameworks used to train and run AI neural network models globally. We demonstrated that it was possible to run the MobileNetV4 object detection/image classification model using ExecuTorch on this platform.

I presented a workshop showing how we optimized ExecuTorch for 32-bit RISC-V at the European RISC-V Summit in Bologna earlier this year. The video is available on YouTube here. The Embecosm team has since refined that workshop, and captured all the material in a new application note.

The application note, “Howto: Bringing up ExecuTorch on a Bare Metal Microcontroller” is now available for download here from the Embecosm website, with the tutorial repositories available on GitHub here. This application note gets you started on bringing up ExecuTorch and then optimizing it to run on small bare metal platforms.

The approach is generic. We use RISC-V as an example, but the overall approach is suitable for any small processor architecture. Since the summer I have given the same workshop at a number of other conferences and to a number of companies. Please contact info@embecosm.com if you would like a presentation of the workshop, either online or hands-on in person.

Feedback on the application note to info@embecosm.com is very welcome. We hope you find it useful.