Bisimulation Prioritized Experience Replay (BPER) introduces a novel approach to enhance value-based reinforcement learning by incorporating a bisimulation metric to prioritize behaviorally relevant transitions. By balancing traditional TD-error with state behavior similarities, BPER improves data diversity and learning efficiency, as demonstrated in a 31-state Grid World and pixel-based environments.

During a single step forward, we compute the covariance tensor and factorize it by Tucker decomposition to generate kernels for a CNN. The kernels were then plugged into the CNN (CovarianceNet) for classification.

We reproduced StyleGAN embedding algorithms, adding further experimentation such as inpainting, super-resolution, colorization, morphing, style transfer, and expression transfer.

Path planning based on Ant Colony Optimization was studied to deal with stuck conditions and speed up convergence.

Successive Adaptive Linear Neural Modeling is a two-step approach based on a Self Organized Map (SOM) and an Adaptative Linear Neuron (Adaline).

This project integrated a pretrained BERT-based model into a Deep Q-Network (DQN) to enhance decision-making in the Hangman word game. The model combines contextual learning from BERT with reinforcement learning, optimizing letter selection via a dynamic reward function based on character frequency and game progression. Explored LoRA fine-tuning, smaller BERT variants, and custom Gym environments improving accuracy from 70% to 75% and achieving a 65% success rate across 113,000 games.

This project aims to fine-tune a BERT-based model, called HangmanNet, for character-level masked language modeling to play the Hangman game. The model predicts hidden characters by leveraging bidirectional context from both sides of a word. Additional techniques, such as incorporating prior character frequency and custom data collators, were explored to improve accuracy, particularly for shorter words.

We developed a multistream Automatic License Plate Recognition (ALPR) system by assembling an Nvidia deepstream pipeline, training 3+ models with the Nvidia TAO toolkit, and periodically sending data streams to a Kafka Apache for further use cases.

We developed a system to segment and count grapes bunches using Mask RCNN and DeepSort Tracker. After that, we extrapolate the counting information to satellite images to generate heat maps that show the number of grapes per parcel in a yield.

We developed a retro video game in 48 hours using Unity3D to measure our expertise and continue improving. Game Jam Topic: Out of Control