Lane follower: Lane-detector (OpenCV) + Object-detector (YOLO5) + CAN-bus

Overview

Lane Follower

This code is for the lane follower, including perception and control, as shown below.

Structure

Environment

  1. Hardware
    • Industrial Camera
    • Intel-NUC(10FNK)
  2. Software
    • Ubuntu18.04
    • Python3.6
    • OpenCV4.2
    • PyTorch1.8.1

    See environment.txt for details.

How to use

A. Offline Testing

The code supports the offline testing, which takes the offline video as input and output the demo video.

python offline_test.py

B. OnLine Testing

The code also supports the online testing, which takes the real-time video streaming from the industrial camera as input and controls the vehicle.

python online_test.py

C. Demo

You can find the offline testing video and the corresponding demo video here [n25o].

demo

Details

Detailed structure

detailed-structure

Code Info

  • offline_test.py --- Offline testing

  • online_test.py --- Online testing

  • basic_function --- Some Basic Function

    • show_img(name, img): Show the image
    • find_files(directory, pattern): Method to find target files in one directory, including subdirectory
    • get_M_Minv(): Get Perspective Transform
    • draw_area(img_origin, img_line, Minv, left_fit, right_fit): Draw the road area in the image
    • draw_demo(img_result, img_bin, img_canny, img_line, img_line_warp, img_bev_result, curvature, distance_from_center, steer): Generate the Demo image
  • lib_camera --- Class for the industrial camera

    • open(): Open the camera
    • grab(): Grab an image from the camera
    • close(): Close the camera
  • mvsdk --- Official lib for the industrial camera

  • lib_can --- Class for the CAN

    • OpenDevice(): Open the CAN device
    • InitCAN(can_idx=0): Init the CAN
    • StartCan(can_idx=0): Start the CAN
    • Send(can_idx, id, frame_len, data): Send messages to CAN
    • Listen(can_idx, id, try_cnt=10): Receive messages from CAN
    • CloseDevice(): Close the CAN device
  • lib_LaneDetector --- Class for the lane detector

    • detect_line(img_input, steer, memory, debug=False): Main Function
    • pre_process(img, debug=False): Image Preprocessing
    • find_line(img, memory, debug=False): Detect the lane using Sliding Windows Methods
    • calculate_curv_and_pos(img_line, left_fit, right_fit): Calculate the curvature & distance from the center
  • lib_ObjectDetector --- Class for the traffic object detector based on YOLO5

    • load_model(): Load Yolo5 model from pytorch hub
    • detect(frame, img_area): Predict and analyze using yolo5
    • class_to_label(idx): Return the corresponding string label for a given label value
    • plot_detections(results, frame): Takes a frame and its results as input, and plots the bounding boxes and label on to the frame
  • lib_vehicle --- Class for the vehicle model and vehicle control

    • steer_cal(curvature, dist_from_center): Calculate the steer according to the curvature of the lane and the distance form the center
    • steer_ctrl(): Control the steer by sending the signal via CAN
    • steer_get(): Get the real steer of the vehicle via the CAN
  • libcontrolcan.so --- DLL for the CAN device

  • libMVSDK.so --- DLL for the industrial camera

Owner
Siqi Fan
Graduate Student @ IA, CAS (2019 ~ now) B.E. @ Shanghai Jiao Tong University (SJTU,2015~2019)
Siqi Fan
This is the codebase for the ICLR 2021 paper Trajectory Prediction using Equivariant Continuous Convolution

Trajectory Prediction using Equivariant Continuous Convolution (ECCO) This is the codebase for the ICLR 2021 paper Trajectory Prediction using Equivar

Spatiotemporal Machine Learning 45 Jul 22, 2022
Live training loss plot in Jupyter Notebook for Keras, PyTorch and others

livelossplot Don't train deep learning models blindfolded! Be impatient and look at each epoch of your training! (RECENT CHANGES, EXAMPLES IN COLAB, A

Piotr Migdał 1.2k Jan 08, 2023
Framework for evaluating ANNS algorithms on billion scale datasets.

Billion-Scale ANN http://big-ann-benchmarks.com/ Install The only prerequisite is Python (tested with 3.6) and Docker. Works with newer versions of Py

Harsha Vardhan Simhadri 132 Dec 24, 2022
TensorFlow 2 AI/ML library wrapper for openFrameworks

ofxTensorFlow2 This is an openFrameworks addon for the TensorFlow 2 ML (Machine Learning) library

Center for Art and Media Karlsruhe 96 Dec 31, 2022
A general framework for inferring CNNs efficiently. Reduce the inference latency of MobileNet-V3 by 1.3x on an iPhone XS Max without sacrificing accuracy.

GFNet-Pytorch (NeurIPS 2020) This repo contains the official code and pre-trained models for the glance and focus network (GFNet). Glance and Focus: a

Rainforest Wang 169 Oct 28, 2022
Experiments on continual learning from a stream of pretrained models.

Ex-model CL Ex-model continual learning is a setting where a stream of experts (i.e. model's parameters) is available and a CL model learns from them

Antonio Carta 6 Dec 04, 2022
UFT - Universal File Transfer With Python

UFT 2.0.0 UFT (Universal File Transfer) is a CLI tool , which can be used to upl

Merwin 1 Feb 18, 2022
A Python script that creates subtitles of a given length from text paragraphs that can be easily imported into any Video Editing software such as FinalCut Pro for further adjustments.

Text to Subtitles - Python This python file creates subtitles of a given length from text paragraphs that can be easily imported into any Video Editin

Dmytro North 9 Dec 24, 2022
GNN-based Recommendation Benchmark

GRecX A Fair Benchmark for GNN-based Recommendation Homepage and Documentation Homepage: Documentation: Paper: GRecX: An Efficient and Unified Benchma

73 Oct 17, 2022
Generate Contextual Directory Wordlist For Target Org

PathPermutor Generate Contextual Directory Wordlist For Target Org This script generates contextual wordlist for any target org based on the set of UR

8 Jun 23, 2021
This repository is for EMNLP 2021 paper: It is Not as Good as You Think! Evaluating Simultaneous Machine Translation on Interpretation Data

InterpretationData This repository is for our EMNLP 2021 paper: It is Not as Good as You Think! Evaluating Simultaneous Machine Translation on Interpr

4 Apr 21, 2022
A modern pure-Python library for reading PDF files

pdf A modern pure-Python library for reading PDF files. The goal is to have a modern interface to handle PDF files which is consistent with itself and

6 Apr 06, 2022
McGill Physics Hackathon 2021: Reaction-Diffusion Models for the Generation of Biological Patterns

DiffuseAnimals: Reaction-Diffusion Models for the Generation of Biological Patterns Introduction Reaction-diffusion equations can be utilized in order

Austin Szuminsky 2 Mar 07, 2022
SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data

SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data Au

14 Nov 28, 2022
EFENet: Reference-based Video Super-Resolution with Enhanced Flow Estimation

EFENet EFENet: Reference-based Video Super-Resolution with Enhanced Flow Estimation Code is a bit messy now. I woud clean up soon. For training the EF

Yaping Zhao 19 Nov 05, 2022
In-place Parallel Super Scalar Samplesort (IPS⁴o)

In-place Parallel Super Scalar Samplesort (IPS⁴o) This is the implementation of the algorithm IPS⁴o presented in the paper Engineering In-place (Share

82 Dec 22, 2022
Implementation of the paper Recurrent Glimpse-based Decoder for Detection with Transformer.

REGO-Deformable DETR By Zhe Chen, Jing Zhang, and Dacheng Tao. This repository is the implementation of the paper Recurrent Glimpse-based Decoder for

Zhe Chen 33 Nov 30, 2022
This repository will be a summary and outlook on all our open, medical, AI advancements.

medical by LAION This repository will be a summary and outlook on all our open, medical, AI advancements. See the medical-general channel in the medic

LAION AI 18 Dec 30, 2022
GPU-accelerated PyTorch implementation of Zero-shot User Intent Detection via Capsule Neural Networks

GPU-accelerated PyTorch implementation of Zero-shot User Intent Detection via Capsule Neural Networks This repository implements a capsule model Inten

Joel Huang 15 Dec 24, 2022
Keras implementation of Normalizer-Free Networks and SGD - Adaptive Gradient Clipping

Keras implementation of Normalizer-Free Networks and SGD - Adaptive Gradient Clipping

Yam Peleg 63 Sep 21, 2022