Utils to quickly evaluate many 🤗 models on the GLUE tasks

Overview

Utils to quickly evaluate many 🤗 models on the GLUE tasks

Setup

wget https://raw.githubusercontent.com/huggingface/transformers/master/examples/pytorch/text-classification/run_glue.py
conda env create -f environment.yaml
conda activate glue-utils

Evaluate multiple models on every glue task:

python runner.py \
    path/to/model_1/ \
    path/to/model_2/ \
    path/to/model_3/ \
    --gpu_idx 1

This will store results in path/to/model_N/glue/[task_name]/, for each model, for each task.

To quickly gather all results, run:

python gather_scores.py \
    path/to/model_1/glue/ \
    path/to/model_2/glue/ \
    path/to/model_3/glue/
Owner
Przemyslaw K. Joniak
Grad student at The University of Tokyo
Przemyslaw K. Joniak
Automatic and platform-independent unpacker for Windows binaries based on emulation

_ _ __ _ __ _ | | | | / / (_) \ \ | | | | | |_ __ | | _ | | _ __ __ _ ___| | _____ _ __

514 Dec 21, 2022
TB Set color display - Add-on for Blender to set multiple objects and material Display Color at once.

TB_Set_color_display Add-on for Blender with operations to transfer name between object, data, materials and action names Set groups of object's or ma

1 Jun 01, 2022
Remove Sheet Protection from .xlsx files. Easily.

🔓 Excel Sheet Unlocker Remove sheet protection from .xlsx files. How to use Run Run the script/packaged executable from the command line. Universal u

Daniel 3 Nov 16, 2022
Manually Install Python 2.7 pip without any problem !

Python2.7_install_pip Manually Install Python 2.7 pip without any problem ! Download installPip.py to your system and Run the code using this Command

Ali Jafari 1 Dec 09, 2021
The Python agent for Apache SkyWalking

SkyWalking Python Agent SkyWalking-Python: The Python Agent for Apache SkyWalking, which provides the native tracing abilities for Python project. Sky

The Apache Software Foundation 149 Dec 12, 2022
Aides to reduce a cheat file with a personal selection of the cheats you want to use.

Retroarch Cheat File Reducer Description Aides to reduce a cheat file with a personal selection of the cheats you want to use. Instructions Copy a sel

1 Jan 09, 2022
Object-data mapper and advanced query manager for non relational databases

Object data mapper and advanced query manager for non relational databases. The data is owned by different, configurable back-end databases and it is

Luca Sbardella 121 Aug 11, 2022
Pulse sequence builder and compiler for q1asm

q1pulse Pulse sequence builder and compiler for q1asm. q1pulse is a simple library to compile pulse sequence to q1asm, the assembly language of Qblox

Sander de Snoo 3 Dec 14, 2022
A program to calculate the are of a triangle. made with Python.

Area-Calculator What is Area-Calculator? Area-Calculator is a program to find out the area of a triangle easily. fully made with Python. Needed a pyth

Chandula Janith 0 Nov 27, 2021
A dashboard for your code. A build system.

NOTICE: THIS REPO IS NO LONGER UPDATED Changes Changes is a build coordinator and reporting solution written in Python. The project is primarily built

Dropbox 763 Sep 09, 2022
This app is to use algorithms to find the root of the equation

In this repository, I made an amazing app with tkinter python language and other libraries the idea of this app is to use algorithms to find the root of the equation I used three methods from numeric

Mohammad Al Jadallah 3 Sep 16, 2022
Repo created for the purpose of adding any kind of programs and projects

Programs and Project Repository A repository for adding programs and projects of any kind starting from beginners level to expert ones Contributing to

Unicorn Dev Community 3 Nov 02, 2022
Rick Astley Language is a rick roll oriented, dynamic, strong, esoteric programming language.

Rick Roll Language / Rick Astley Language A rick roll oriented, dynamic, strong, esoteric programming language. Prolegomenon The reasons that I made t

Rick Roll Programming Language 658 Jan 09, 2023
Time tracking program that will format output to be easily put into Gitlab

time_tracker Time tracking program that will format output to be easily put into Gitlab. Feel free to branch and use it yourself! Getting Started Clon

Jake Strasler 2 Oct 13, 2022
Advanced Variable Manager {AVM} [0.8.0]

Advanced Variable Manager {AVM} [0.8.0] By Grosse pastèque#6705 WARNING : This modules need some typing modifications ! If you try to run it without t

Big watermelon 1 Dec 11, 2021
A lightweight solution for local Particle development.

neopo A lightweight solution for local Particle development. Features Builds Particle projects locally without any overhead. Compatible with Particle

Nathan Robinson 19 Jan 01, 2023
An example file showing a simple endpoints like a login/logout function and maybe some others.

Flask API Example An example project showing a simple endpoints like a login/logout function and maybe some others. How to use: Open up your IDE (or u

Kevin 1 Oct 27, 2021
Kunai Shitty Raider Leaked LMFAO

Kunai-Raider-Leaked Kunai Shitty Raider Leaked LMFA

5 Nov 24, 2021
The Official Jaseci Code Repository

Jaseci Release Notes Version 1.2.2 Updates Added new built-ins for nodes and edges (context, info, and details) Fixed dot output Added reset command t

136 Dec 20, 2022
Commodore 64 OS running on Atari 8-bit hardware

This is the Commodre 64 KERNAL, modified to run on the Atari 8-bit line of computers. They're practically the same machine; why didn't someone try this 30 years ago?

Nick Bensema 133 Nov 12, 2022