Utils to quickly evaluate many 🤗 models on the GLUE tasks

Overview

Utils to quickly evaluate many 🤗 models on the GLUE tasks

Setup

wget https://raw.githubusercontent.com/huggingface/transformers/master/examples/pytorch/text-classification/run_glue.py
conda env create -f environment.yaml
conda activate glue-utils

Evaluate multiple models on every glue task:

python runner.py \
    path/to/model_1/ \
    path/to/model_2/ \
    path/to/model_3/ \
    --gpu_idx 1

This will store results in path/to/model_N/glue/[task_name]/, for each model, for each task.

To quickly gather all results, run:

python gather_scores.py \
    path/to/model_1/glue/ \
    path/to/model_2/glue/ \
    path/to/model_3/glue/
Owner
Przemyslaw K. Joniak
Grad student at The University of Tokyo
Przemyslaw K. Joniak
Irrigation Component V4 providing support for a custom card

Irrigation Component V4 This release sees the delivery of a custom card https://github.com/petergridge/irrigation_card to render the program options s

12 Oct 28, 2022
A python script developed to process Windows memory images based on triage type.

Overview A python script developed to process Windows memory images based on triage type. Requirements Python3 Bulk Extractor Volatility2 with Communi

CrowdStrike 245 Nov 24, 2022
Petuhlang is a joke-like language, based on Python.

Petuhlang is a joke-like language, based on Python. It updates builtins to make a new syntax based on operators rewrite.

DenyS 9 Jun 19, 2022
A feed generator. Currently supports generating RSS feeds from Google, Bing, and Yahoo news.

A feed generator. Currently supports generating RSS feeds from Google, Bing, and Yahoo news.

Josh Cardenzana 0 Dec 13, 2021
Automates the fixing of problems reported by yamllint by parsing its output

yamlfixer yamlfixer automates the fixing of problems reported by yamllint by parsing its output. Usage This software automatically fixes some errors a

OPT Nouvelle Caledonie 26 Dec 26, 2022
A simple solution for water overflow problem in Python

Water Overflow problem There is a stack of water glasses in a form of triangle as illustrated. Each glass has a 250ml capacity. When a liquid is poure

Kris 2 Oct 22, 2021
A simple projects to help your seo optimizing has been written with python

python-seo-projects it is a very simple projects to help your seo optimizing has been written with python broken link checker with python(it will give

Amirmohammad Razmy 3 Dec 25, 2021
Distributed behavioral experiments

Autopilot Docs Paper Forum Hardware Autopilot is a Python framework for performing complex, hardware-intensive behavioral experiments with swarms of n

70 Dec 14, 2022
tg-nearby Trilateration of nearby Telegram users as described in my corresponding article.

tg-nearby Trilateration of nearby Telegram users as described in my corresponding article. Setup If you want to toy with the code in this repository

Maximilian Jugl 75 Dec 26, 2022
Free and open source qualitative research tool

Taguette A spin on the phrase "tag it!", Taguette is a free and open source qualitative research tool that allows users to: Import PDFs, Word Docs (.d

Remi Rampin 48 Jan 02, 2023
A hackers attempt at an MVP anki plugin

my anki plugin if you have found this by accident, you should probably run away this is nothing more than a hackers attempt at an MVP anki plugin I re

Chris Hall 1 Nov 02, 2021
A (hopefully) considerably copious collection of classical cipher crackers

ClassicalCipherCracker A (hopefully) considerably copious collection of classical cipher crackers Written in Python3 (and run with PyPy) TODOs Write a

Stanley Zhong 2 Feb 22, 2022
Movie recommend community

README 0. 초록 1) 목적 사용자의 Needs를 기반으로 영화를 추천해주는 커뮤니티 서비스 구현 2) p!ck 서비스란? "pick your taste!" 취향대로 영화 플레이리스트(이하 서비스 내에서의 명칭인 '바스켓'이라 함)를 만들고, 비슷한 취향을 가진

2 Dec 08, 2021
🌍💉 Global COVID-19 vaccination data at the regional level.

COVID-19 vaccination data at subnational level. To ensure its officiality, the source data is carefully verified.

sociepy 61 Sep 21, 2022
TB Set color display - Add-on for Blender to set multiple objects and material Display Color at once.

TB_Set_color_display Add-on for Blender with operations to transfer name between object, data, materials and action names Set groups of object's or ma

1 Jun 01, 2022
Scitizen - Help scientific research for the benefit of mankind and humanity 🔬

Scitizen - Help scientific research for the benefit of mankind and humanity 🔬 Scitizen has been built from the ground up to give everyone the possibi

Pierre CORBEL 21 Mar 08, 2022
HungryBall to prosta gra, w której gracz wciela się w piłkę.

README POLSKI Opis gry HungryBall to prosta gra, w której gracz wciela się w piłkę. Sterowanie odbywa się za pomocą przycisków w, a, s i d lub opcjona

Karol 1 Nov 24, 2021
Audio-analytics for music-producers! Automate tedious tasks such as musical scale detection, BPM rate classification and audio file conversion.

Click here to be re-directed to the Beat Inspect Streamlit Web-App You are a music producer? Let's get in touch via LinkedIn Fundamental Analytics for

Stefan Rummer 11 Dec 27, 2022
Multi-Probe Attention for Semantic Indexing

Multi-Probe Attention for Semantic Indexing About This project is developed for the topic of COVID-19 semantic indexing. Directories & files A. The di

Jinghang Gu 1 Dec 18, 2022
adbsync - An ADB syncing helper

adbsync - An ADB syncing helper What's this? Everytime I wanted to make a backup of my phone, or restore those files onto it, I had to use everytime t

Giovanni Gualtieri 3 Aug 05, 2022