This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
A hashtag check python module

A hashtag check python module

Fayas Noushad 3 Aug 10, 2022
Generate obfuscated meterpreter shells

Generator Evade AV with obfuscated payloads Installation must install dotnet prior to running the script with net45 Running ./generator.py -ip Your-I

Fawaz Al-Mutairi 219 Nov 28, 2022
This is a partial and quick and dirty proof of concept implementation of the following specifications to configure a tor client to use trusted exit relays only.

This is a partial and quick and dirty proof of concept implementation of the following specifications to configure a tor client to use trusted exit re

22 Nov 09, 2022
Make your own huge Wordlist with advanced options

#It's my first tool i hope to be useful for everyone, Make your own huge Wordlist with advanced options, You need python3 to run this tool, If you hav

0.1Arafa 6 Dec 08, 2022
A dynamic multi-STL, multi-process OpenSCAD build system with autoplating support

scad-build This is a multi-STL OpenSCAD build system based around GNU make. It supports dynamic build targets, intelligent previews with user-defined

Jordan Mulcahey 1 Dec 21, 2021
CVE-2021-22205& GitLab CE/EE RCE

Vuln Impact An issue has been discovered in GitLab CE/EE affecting all versions starting from 11.9. GitLab was not properly validating image files tha

Al1ex 213 Dec 30, 2022
PKUAutoElective for 2021 spring semester

PKUAutoElective 2021 Spring Version Update at Mar 7 15:28 (UTC+8): 修改了 get_supplement 的 API 参数,已经可以实现课程列表页面的正常跳转,请更新至最新 commit 版本 本项目基于 PKUAutoElectiv

Zihan Mao 84 Sep 09, 2022
Chapter 1 of the AWS Cookbook

Chapter 1 - Security Set and export your default region: export AWS_REGION=us-east-1 Set your AWS ACCOUNT ID:: AWS_ACCOUNT_ID=$(aws sts get-caller-ide

AWS Cookbook 30 Nov 27, 2022
Nmap automated port scanner written in Python

port-scanner Nmap automated port scanner written in Python. USE: Clone the module Import the module: from portscanModule import portscanner Use: ports

Brayden Karnes 1 Dec 03, 2021
Hikvision 流媒体管理服务器敏感信息泄漏

Hikvisioninformation Hikvision 流媒体管理服务器敏感信息泄漏 Options optional arguments: -h, --help show this help message and exit -u url, --url url

Henry4E36 13 Nov 09, 2022
An advanced multi-threaded, multi-client python reverse shell for hacking linux systems

PwnLnX An advanced multi-threaded, multi-client python reverse shell for hacking linux systems. There's still more work to do so feel free to help out

0xTRAW 212 Dec 24, 2022
Ingest GreyNoise.io malicious feed for CVE-2021-44228 and apply null routes

log4j-nullroute Quick script to ingest IP feed from greynoise.io for log4j (CVE-2021-44228) and null route bad addresses. Works w/Cisco IOS-XE and Ari

Ryan 5 Sep 12, 2022
pwncat module that automatically exploits CVE-2021-4034 (pwnkit)

pwncat_pwnkit Introduction The purpose of this module is to attempt to exploit CVE-2021-4034 (pwnkit) on a target when using pwncat. There is no need

Dana Epp 33 Jul 01, 2022
Aiminsun 165 Dec 21, 2022
Safe Policy Optimization with Local Features

Safe Policy Optimization with Local Feature (SPO-LF) This is the source-code for implementing the algorithms in the paper "Safe Policy Optimization wi

Akifumi Wachi 6 Jun 05, 2022
Raphael is a vulnerability scanning tool based on Python3.

Raphael Raphael是一款基于Python3开发的插件式漏洞扫描工具。 Raphael is a vulnerability scanning too

b4zinga 5 Mar 21, 2022
This collection of tools that makes it easy to secure and/or obfuscate messages, files, and data.

Scrambler App This collection of tools that makes it easy to secure and/or obfuscate messages, files, and data. It leverages encryption tools such as

Mystic 2 Aug 31, 2022
Denial Attacks by Various Methods

Denial Service Attack Denial Attacks by Various Methods IIIIIIIIIIIIIIIIIIII PPPPPPPPPPPPPPPPP VVVVVVVV VVVVVVVV I::

Baris Dincer 9 Nov 26, 2022
Tools ini digunakan untuk krekk pacebuk:v

E-Crack By Aang-XD Fitur Login • Login via token facebook • Login via cookie facebook Install On Termux $ pkg update && pkg upgrade $ pkg install pyth

Aang Ardiansyah-XD 2 Dec 24, 2021
Log4j minecraft with python

log4jminecraft This code DOES NOT promote or encourage any illegal activities! The content in this document is provided solely for educational purpose

David Bombal 154 Dec 24, 2022