Lane follower: Lane-detector (OpenCV) + Object-detector (YOLO5) + CAN-bus

Overview

Lane Follower

This code is for the lane follower, including perception and control, as shown below.

Structure

Environment

  1. Hardware
    • Industrial Camera
    • Intel-NUC(10FNK)
  2. Software
    • Ubuntu18.04
    • Python3.6
    • OpenCV4.2
    • PyTorch1.8.1

    See environment.txt for details.

How to use

A. Offline Testing

The code supports the offline testing, which takes the offline video as input and output the demo video.

python offline_test.py

B. OnLine Testing

The code also supports the online testing, which takes the real-time video streaming from the industrial camera as input and controls the vehicle.

python online_test.py

C. Demo

You can find the offline testing video and the corresponding demo video here [n25o].

demo

Details

Detailed structure

detailed-structure

Code Info

  • offline_test.py --- Offline testing

  • online_test.py --- Online testing

  • basic_function --- Some Basic Function

    • show_img(name, img): Show the image
    • find_files(directory, pattern): Method to find target files in one directory, including subdirectory
    • get_M_Minv(): Get Perspective Transform
    • draw_area(img_origin, img_line, Minv, left_fit, right_fit): Draw the road area in the image
    • draw_demo(img_result, img_bin, img_canny, img_line, img_line_warp, img_bev_result, curvature, distance_from_center, steer): Generate the Demo image
  • lib_camera --- Class for the industrial camera

    • open(): Open the camera
    • grab(): Grab an image from the camera
    • close(): Close the camera
  • mvsdk --- Official lib for the industrial camera

  • lib_can --- Class for the CAN

    • OpenDevice(): Open the CAN device
    • InitCAN(can_idx=0): Init the CAN
    • StartCan(can_idx=0): Start the CAN
    • Send(can_idx, id, frame_len, data): Send messages to CAN
    • Listen(can_idx, id, try_cnt=10): Receive messages from CAN
    • CloseDevice(): Close the CAN device
  • lib_LaneDetector --- Class for the lane detector

    • detect_line(img_input, steer, memory, debug=False): Main Function
    • pre_process(img, debug=False): Image Preprocessing
    • find_line(img, memory, debug=False): Detect the lane using Sliding Windows Methods
    • calculate_curv_and_pos(img_line, left_fit, right_fit): Calculate the curvature & distance from the center
  • lib_ObjectDetector --- Class for the traffic object detector based on YOLO5

    • load_model(): Load Yolo5 model from pytorch hub
    • detect(frame, img_area): Predict and analyze using yolo5
    • class_to_label(idx): Return the corresponding string label for a given label value
    • plot_detections(results, frame): Takes a frame and its results as input, and plots the bounding boxes and label on to the frame
  • lib_vehicle --- Class for the vehicle model and vehicle control

    • steer_cal(curvature, dist_from_center): Calculate the steer according to the curvature of the lane and the distance form the center
    • steer_ctrl(): Control the steer by sending the signal via CAN
    • steer_get(): Get the real steer of the vehicle via the CAN
  • libcontrolcan.so --- DLL for the CAN device

  • libMVSDK.so --- DLL for the industrial camera

Owner
Siqi Fan
Graduate Student @ IA, CAS (2019 ~ now) B.E. @ Shanghai Jiao Tong University (SJTU,2015~2019)
Siqi Fan
Multiple-criteria decision-making (MCDM) with Electre, Promethee, Weighted Sum and Pareto

EasyMCDM - Quick Installation methods Install with PyPI Once you have created your Python environment (Python 3.6+) you can simply type: pip3 install

Labrak Yanis 6 Nov 22, 2022
September-Assistant - Open-source Windows Voice Assistant

September - Windows Assistant September is an open-source Windows personal assis

The Nithin Balaji 9 Nov 22, 2022
Denoising Diffusion Probabilistic Models

Denoising Diffusion Probabilistic Models This repo contains code for DDPM training. Based on Denoising Diffusion Probabilistic Models, Improved Denois

Alexander Markov 7 Dec 15, 2022
Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset (CVPR'19)

Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset (CVPR'19) Tianyu Wang*, Xin Yang*, Ke Xu, Shaozhe Chen, Qiang Zhang, Ry

Steve Wong 177 Dec 01, 2022
Pytorch implementation for Patient Knowledge Distillation for BERT Model Compression

Patient Knowledge Distillation for BERT Model Compression Knowledge distillation for BERT model Installation Run command below to install the environm

Siqi 180 Dec 19, 2022
Lux AI environment interface for RLlib multi-agents

Lux AI interface to RLlib MultiAgentsEnv For Lux AI Season 1 Kaggle competition. LuxAI repo RLlib-multiagents docs Kaggle environments repo Please let

Jaime 12 Nov 07, 2022
This is a collection of our NAS and Vision Transformer work.

AutoML - Neural Architecture Search This is a collection of our AutoML-NAS work iRPE (NEW): Rethinking and Improving Relative Position Encoding for Vi

Microsoft 828 Dec 28, 2022
Solutions of Reinforcement Learning 2nd Edition

Solutions of Reinforcement Learning, An Introduction

YIFAN WANG 1.4k Dec 30, 2022
Explainability for Vision Transformers (in PyTorch)

Explainability for Vision Transformers (in PyTorch) This repository implements methods for explainability in Vision Transformers

Jacob Gildenblat 442 Jan 04, 2023
Husein pet projects in here!

project-suka-suka Husein pet projects in here! List of projects mysejahtera-density. Generate resolution points using meshgrid and request each points

HUSEIN ZOLKEPLI 47 Dec 09, 2022
Official implementation of the article "Unsupervised JPEG Domain Adaptation For Practical Digital Forensics"

Unsupervised JPEG Domain Adaptation for Practical Digital Image Forensics @WIFS2021 (Montpellier, France) Rony Abecidan, Vincent Itier, Jeremie Boulan

Rony Abecidan 6 Jan 06, 2023
Official repository of "DeepMIH: Deep Invertible Network for Multiple Image Hiding", TPAMI 2022.

DeepMIH: Deep Invertible Network for Multiple Image Hiding (TPAMI 2022) This repo is the official code for DeepMIH: Deep Invertible Network for Multip

Junpeng Jing 67 Nov 22, 2022
Hierarchical User Intent Graph Network for Multimedia Recommendation

Hierarchical User Intent Graph Network for Multimedia Recommendation This is our Pytorch implementation for the paper: Hierarchical User Intent Graph

6 Jan 05, 2023
Codes for TIM2021 paper "Anchor-Based Spatio-Temporal Attention 3-D Convolutional Networks for Dynamic 3-D Point Cloud Sequences"

Codes for TIM2021 paper "Anchor-Based Spatio-Temporal Attention 3-D Convolutional Networks for Dynamic 3-D Point Cloud Sequences"

Intelligent Robotics and Machine Vision Lab 4 Jul 19, 2022
Repository for Multimodal AutoML Benchmark

Benchmarking Multimodal AutoML for Tabular Data with Text Fields Repository for the NeurIPS 2021 Dataset Track Submission "Benchmarking Multimodal Aut

Xingjian Shi 44 Nov 24, 2022
A gesture recognition system powered by OpenPose, k-nearest neighbours, and local outlier factor.

OpenHands OpenHands is a gesture recognition system powered by OpenPose, k-nearest neighbours, and local outlier factor. Currently the system can iden

Paul Treanor 12 Jan 10, 2022
PyTorch implementation of Glow

glow-pytorch PyTorch implementation of Glow, Generative Flow with Invertible 1x1 Convolutions (https://arxiv.org/abs/1807.03039) Usage: python train.p

Kim Seonghyeon 433 Dec 27, 2022
Autonomous Ground Vehicle Navigation and Control Simulation Examples in Python

Autonomous Ground Vehicle Navigation and Control Simulation Examples in Python THIS PROJECT IS CURRENTLY A WORK IN PROGRESS AND THUS THIS REPOSITORY I

Joshua Marshall 14 Dec 31, 2022
robomimic: A Modular Framework for Robot Learning from Demonstration

robomimic [Homepage]   [Documentation]   [Study Paper]   [Study Website]   [ARISE Initiative] Latest Updates [08/09/2021] v0.1.0: Initial code and pap

ARISE Initiative 178 Jan 05, 2023
Additional environments compatible with OpenAI gym

Decentralized Control of Quadrotor Swarms with End-to-end Deep Reinforcement Learning A codebase for training reinforcement learning policies for quad

Zhehui Huang 40 Dec 06, 2022