
# Introduction

This project approaches creating a proposed neuromorphic computing method in Spiking Neural Networks (SNNs). SNNs are built upon biomimicry of spiking activity within the human brain and emulate it using memristor technology and modern machine learning theory. We aim to create novel, increasingly power-efficient approaches to AI inference. Developers will be split into hardware and software teams where the software team will approach training the model and modeling data to fit a spiking nature, and the hardware team will construct the required memristor array and functional “processor” used in the SNN.

We are looking for software developers with experience in neural network architectures, data manipulation, and hardware developers with experience in PCB Fabrication and the fundamentals of creating computer systems. Experience with SNNTorch or general knowledge of Memristors is a plus. If you have any questions or would like to reach out, feel free to email vraj.prajapati@mail.utoronto.ca or message @vpro on Discord.

[Proposal](https://drive.google.com/open?id=14ONsf54LJXB3w6cbgoM8W0PpJtcq15Sl)


Build a spiking neural network with resistance RAM.

Hardware Team

Software Team

Spiking Neural Networks


# Introduction

The stock market has always played a huge role in the global economy. That is why it will be meaningful to see how artificial intelligence can be applied in stock price prediction. As beginners to AI, this was an excellent project to start with. For AI projects, data is integral to its success and often it could be hard to obtain reliable data. However, the stock market has many different datasets that are readily available. Both from an interest and practicality perspective, stock price prediction was a great project for us who want to learn more about applications of AI in the real world.

## Goals

- Learn the basics of artificial intelligence and machine learning
- Apply this knowledge to the stock price prediction project using online articles as aid
- Have fun learning something new and interesting

## Learnings and Tutorials

- [Machine Learning by Stanford University](https://www.coursera.org/learn/machine-learning)
- [Neural Networks and Deep Learning](https://www.coursera.org/learn/neural-networks-deep-learning)
- [Predicting Stock Trend Using Deep Learning](https://dzone.com/articles/predicting-stock-trend-using-deep-learning)

## Methodologies and Tools

- Dataset: NSE Tata Global stock, Apple stock
- Recurrent Neural Network (RNN)
- Pandas: preprocessing
- Keras: LSTM, Adam optimizer

## Result

The model is quite accurate in the beginning but the accuracy decreases as time increases

![result](/imgs/projects/stock_price_prediction/Picture3.png)


Developer

Stock Price Prediction


# Description

Last year at UTMIST, the Virtual Creatures Project successfully managed to evolve and train an impressively diverse range of walking and swimming creatures, all just using a simple evolutionary algorithm from 1994. This year, we will take this project to the next level by introducing **intelligent teamwork and collaboration** between these evolved creatures, using **NEAT** and modern **multi-agent reinforcement learning** techniques.

Developers best suited for this project should have a strong interest in evolutionary algorithms and reinforcement learning, have experience with Unity and C#, and be committed to 3-6 hours per week. Email anoop.reh@gmail.com or efetoronto@gmail.com (or DM “Anoop R.” or “Efe T.” in the UTMIST Discord) if you’re interested!

[Proposal](https://drive.google.com/file/d/12kgRlOYSGMxtgv5bYpyUlp1U4TUWPrRh/view?usp=sharing)

# Developer Requirements

Our team is divided into the following three subteams, each of which have their own respective requirements:

The **Evolution Subteam** is mainly responsible for evolving creatures using ml-agents and NEAT. We are looking for **4 developers** that ideally have prior experience with or a strong understanding of:
- Modern multi-agent reinforcement learning algorithms
- Evolutionary algorithms, and especially the NEAT algorithm
- Implementing and performing operations on complex graph structures
- Working on large codebases in C#

The **Environment Subteam** is mainly responsible for developing the soccer environment in Unity and interfacing it with the creatures. We are looking for **2 developers** that ideally have prior experience with or a strong understanding of:
- Developing 3D physics-based games with Unity and C#
- Implementing and performing operations on complex graph structures

The **Visualization Subteam** is mainly responsible for developing informative visualizations of both the training process as well as the physical representations of the creatures themselves. We are looking for **2 developers** that ideally have prior experience with or a strong understanding of:
- Insightful data analysis and visualization
- 3D graphics development in Unity (working with shaders, lighting, VFX, etc.)
- UI design and implementation in Unity
- Creating simple 2D and 3D assets (models and textures)


Evolving teamwork between virtual creatures.

Evolution Subteam

Environment Subteam

Visualization Subteam

Evolving Virtual Creatures... PART II!


# Introduction

["The Virtual Creatures Project" Discord](https://discord.gg/8RHYfDZecx)

This project consists of a re-implementation and novel interpretation of Karl Sims’ 1994 paper titled "Evolving Virtual Creatures". The paper presents a method for generating, evaluating, and mutating virtual 3D genotypes, which results in a remarkable diversity of solutions to various tasks. We aim to recreate and expand upon the success of the original publication by using modern engines, asynchronous cloud training, and reinforcement learning theory, focusing on optimizing for 3D swimming. Developers will comprehend academic papers, implement algorithms from the fields of evolutionary computation and reinforcement learning, and build a functioning interface in Unity.

We are looking for developers who love machine learning and have experience in C#, Python, or C. Experience/interest in reinforcement learning, Unity, and/or evolutionary algorithms is also a plus. Email andrew.magnuson@mail.utoronto.ca, or DM “Andrew Magnuson” in the UTMIST Discord if you’re interested!

[Proposal](https://1drv.ms/b/s!ApikpRh7lFXpkUHFx9iZlb9KaZ9N)


Reimagining Karl Sims 1994 'Evolving Virtual Creatures' paper.

Developer Needed

Virtual Creatures


###### [![youtube video](http://img.youtube.com/vi/g200QGSReaw/0.jpg)](https://www.youtube.com/watch?v=g200QGSReaw "WallStreetBots: Stock Price Prediction")

# Introduction

Creating a stock trading AI is far more sophisticated than predicting future prices based on numerical methods such as momentum or machine learning methods such as LSTM or transformers. While the latter involves pure data analysis and machine learning, the former covers much greater scopes, including price prediction, portfolio management strategies, market condition analysis, stock sentiment analysis, data communications, and server deployment. In this project, we aim to create an AI that is practically useful—yes, that means it makes money. We will try a variety of ML algorithms, from simple RNN to NLP, and hopefully reinforcement learning—and in the end, our work will be deployed as a bot to trade with paper money. Will it be able to make money? Let’s find out!

# Framework

**Dataset**

To obtain real-time data such as order types, store stock information, and simulate the trading environment realistically, we built our framework on top of the Alpaca paper trade API. Whenever a user places an order at wallstreetbots.org, the same information is updated at Alpaca paper trade and vice versa.

![Alpaca-API](/imgs/projects/wallstreetbots/wsb-alpaca.png)

**Webapp**

The web app was built using the core technologies of Docker, Python+Django, and PostgreSQL. The web app is roughly split into the basic CRUD functionality of the dashboard for showing portfolios and entering orders, and the actual stock trading engine that makes the decisions. ML strategies are fed into the generic pipeline that is called by the web app every day to automatically rebalance for each user. Each new machine learning strategy inherits from a generic strategy and overrides with a different rebalancing method. This ensures that adding new strategies is as easy as possible.

**Functionalities**

Our ML models predict next-day closing/open stock prices by probing into both traditional market data and sentiment analysis from real-time news headlines and Reddit discussions. The predictions are then interpreted by our portfolio balancing pipeline, where we place trade orders based on the differences between the optimal weights and our current portfolio weights each day.

# Architecture

![WallStreetBots Architecture](/imgs/projects/wallstreetbots/wsb-architecture.png)

The image above summarizes the entire pipeline of WallStreetBots.

Besides the market data obtained from Alpaca, we also pulled textual information for sentiment analysis from:
- NYT News Archive
- FinViz News Headlines
- r/wallstreetbets Daily Discussion Comments
- RSS feeds from various news sources

These textual information are combined with traditional market data when fed into our distilled LSTM model. The input features are summarized in the table below:

![Input features for WallStreetBots](/imgs/projects/wallstreetbots/wsb-features.png)

# Results

![Results of models tested](/imgs/projects/wallstreetbots/wsb-model-results.png)
![Performance of our best model -- Distilled LSTM](/imgs/projects/wallstreetbots/wsb-graph-results.png)

# Future Works

Building on top of the success of this project in 2022, we will be expanding our framework to the market of cryptocurrencies, where quantitative analysis is expected to yield much better correlations. We are expecting to perform price predictions for the 14 most common cryptos by the end of this year. For more information regarding our proposal and recruitment, please visit https://utmist.gitlab.io/projects/wallstreetbots2/.

We also plan to improve our textual analysis pipeline by incorporating Twitter data, which has been shown to influence stock prices over a wide range of time spans. Our NLP subteam will be experimenting with novel methodologies this year to understand the correlations between tweets and their temporal influence on stock/crypto prices.


Price prediction and portfolio/market analysis using NLP and reinforcement learning.

NLP Team

Prediction Model Team

WallStreetBots: Stock Price Prediction


###### [![markdown video](http://img.youtube.com/vi/g200QGSReaw/0.jpg)](https://www.youtube.com/watch?v=g200QGSReaw "markdown video")

# Introduction

The volatile nature of cryptocurrency prices and its being driven by supply and demand makes machine learning an appropriate tool to predict their prices.

Crypto price prediction depends on knowing people’s beliefs about them and how these sentiments will affect trading behaviors based on the current market conditions. This project touches on both aspects of the task. First, we will scrape data from different sources (including Twitter, Reddit, and The New York Times) to create NLP sentiment indicators describing how society feels about crypto. We will then combine these indicators with intuitive trading predictor features based on market data to use as inputs to state-of-the-art machine learning techniques to predict the future price of cryptocurrencies. Our predictions will form the basis for an automatic trading strategy to be tested on an online platform already developed by WallStreetBots last year.

We are looking for developers to work in the three subteams of WallStreetBots: (1) market predictor feature engineering team (finding intuitive predictors based on market data to use as inputs to the final prediction model), (2) prediction model team (designing and fine-tuning specific deep learning models to predict the crypto price based on input features), (3) NLP team (designing clean NLP sentiment indicators to use as inputs to the final prediction model). We look forward to your participation in the project.

Developers should be proficient at Python and have experience with basic machine learning frameworks/libraries. Knowledge and interest in trading is a plus. Time commitment is 1-3 hours per week.

[Proposal](https://1drv.ms/b/s!ApikpRh7lFXpkUgCFHpeswyDHCh6)


Continuation of 2021’s WallStreetBots with a twist -- crypto price prediction.

Market Predictor Feature Engineering Team

WallStreetBots2


# Introduction

This project presents a unique opportunity for developers to contribute to environmental sustainability while gaining valuable experience in machine learning for audio applications.

Wind energy in Canada accounts for 3.5 percent of electricity generation and is the second most important renewable energy source in the region. However, during the commissioning of a new wind turbine generation facility in Ontario, provincial regulations require that a study be performed to assess the noise pollution impact on surrounding residents. The study involves collecting thousands of sound level datapoints over the course of months, using these to establish sound level as a function of wind speed with turbine operation and classify datapoints on audibility. The volume of data required to be collected under various weather and operating conditions, in addition to stringent data quality requirements, means that the measurement campaign is long and subsequent analysis is very rigorous and costly.

To automate this process, UTMIST will be working with the sound consulting company [Aercoustics](https://aercoustics.com/) to provide an efficient ML solution. The timeline of this project is expected to span across the Winter and Summer term. We are recruiting both a director and developers who can commit at least 6 months to this project. You will be expected to design the ML implementation plans based on Aercoustics’ business demands, deliver a minimum viable product (with a command line interface), and meet their regular KPIs.


Providing an ML solution for the turbine audibility problem to Aercoustics Ltd.

Director Needed

Developers Needed

Wind Turbine Audibility - Aercoustics Collaboration


# Introduction

Recognizing handwritten text is one of the most well-known and long-studied tasks in machine learning. In fact, when the CNN was created by Yann LeCun, recognizing handwritten digits was its first task, with the MNIST dataset being used. However, in this project, director Justin Tran and his team present a solution to this problem of Handwritten Text Recognition (HTR) that significantly outperforms prior state-of-the-art models, all without the use of convolutional neural networks.

Recognizing written text is integral to digitizing documents in industries like healthcare, insurance, and banking. However, the biggest challenge is in the high variation of writing styles, and though many software applications already possess HTR features, they are far from perfect. Active research is still ongoing in this field, and with continuous advancements comes many more streamlined processes across numerous fields that rely upon this technology.

# Framework

**Dataset**

![Sample image in dataset](/imgs/projects/handwriting_recognition/handwriting-dataset.png)

The IAM dataset was used to train and test the model. It includes 13,353 images of lines of handwritten text, which are used as the inputs to the HTR algorithm (word and document levels are also provided). Given a document, the bounding boxes for them are also provided and were used to train the segmentation step of the model.

# Architecture

**Network Structure (Segmentation)**

![Segmenting letters](/imgs/projects/handwriting_recognition/handwriting-vertical-segmentation.png)

The algorithm used to perform word segmentation is based on the paper ‘Scale Space Techniques for Word Segmentation in Handwritten Documents’. The algorithm proposed in the paper considers the scale-space behavior of blobs in images containing lines. The underlying concept of this algorithm is the utilization of Gaussian filters to generate an image’s scale space, which involves developing a family of signals where details are progressively removed.

In order to accurately segment the lines within a document, projection profile techniques were utilized. The vertical projection profile is defined by summing the pixel intensity values along a particular row. The lines are identified by determining local peaks in the projection profiles after applying Gaussian smoothing to deal with false positives. After line segmentation, second-order differential Gaussian filters are applied along both orientations of the image in order to form a blob (a connected region) representation of the image. By convolving the image with the second-order differential Gaussian, a scale-space representation is created. The blobs in this representation appear brighter or darker than the background after this convolution. By altering the parameters of the filters, the blobs will transition from characters to words.

**Network Structure (Classification)**

![Classification network structure](/imgs/projects/handwriting_recognition/handwriting-model-cls.png)

The architecture used was based on the paper “TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models” by Li et al. [2]. The architecture for classification was implemented after the segmentation step. It is composed of an encoder and a decoder. The image is first resized and broken into a batch of smaller 16x16 patches, which are then flattened to form what we call patch embeddings. Based on the position of each patch, they are also given a position embedding. These inputs are then passed into a stack of encoders composed of a multi-head self-attention module and a feed-forward network.

The outputs of the encoder are then fed into a decoder architecture, which is almost the same as the encoder architecture. The difference lies in adding an “encoder-decoder” attention module between the two original encoder components. The embedding from the decoder is then projected into the size of the vocabulary, and after applying the softmax function and beam search, the final output is determined.

# Results

![Loss Curve](/imgs/projects/handwriting_recognition/handwriting-results.png)

Overall, test documents were able to be read fairly well. The test documents were read with an average character error of 0.17. For comparison, when trying with a CRNN approach, the character error was 1.16. For reference, if the model had output the word “Hyundai” when the label was “Honda,” the character error would be 0.6. During classification training, the final average training cross-entropy loss was 0.118, and the final average validation loss was 0.386. However, it should be noted that the data used were either pre-segmented text lines in the case of classification training or documents where text lines were reasonably on level, i.e., parallel to the page and not slanted. The model does not perform as well when the text lines are not on level due to the limitations of the segmentation algorithm.

# Links

[Proposal](https://1drv.ms/b/s!ApikpRh7lFXpjwxG43oAHxGoaMo1?e=awwawk)

[Medium Article](https://utorontomist.medium.com/handwriting-recognition-using-deep-learning-14ec078872b0)


Recognizing Handwritten Text in Physical Documents

Justin Tran

Handwriting Recognition Using Deep Learning


  # example

  welcome **markdown example**. this is a test to check if the markdown can render

  ## list

  - hi
  - hello
  - hey

  ## team

  ![team](/team/default.svg)  
  [team - LinkedIn](https://linkedin.com/in/example)  
  [team - Twitter](https://twitter.com/example)

  ## images
  ###### ![img 1](/imgs/projects/project1/project1v1.svg)  
  ###### [![markdown video](http://img.youtube.com/vi/dQw4w9WgXcQ/0.jpg)](https://www.youtube.com/watch?v=dQw4w9WgXcQ "markdown video")


John Doe

project example

  # example

  welcome **markdown example**. this is a test to check if the markdown can render

  ## list

  - hi
  - hello
  - hey

  ## team

  ![team](/team/default.svg)  
  [team - LinkedIn](https://linkedin.com/in/example)  
  [team - Twitter](https://twitter.com/example)

  ## images

  ###### ![img 1](/imgs/projects/project1/project1v1.svg)  
  ###### [![markdown video](http://img.youtube.com/vi/dQw4w9WgXcQ/0.jpg)](https://www.youtube.com/watch?v=dQw4w9WgXcQ "markdown video")