This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
BoobSnail allows generating Excel 4.0 XLM macro. Its purpose is to support the RedTeam and BlueTeam in XLM macro generation.

Follow us on Twitter! BoobSnail BoobSnail allows generating XLM (Excel 4.0) macro. Its purpose is to support the RedTeam and BlueTeam in XLM macro gen

STM Cyber 232 Nov 21, 2022
evtx-hunter helps to quickly spot interesting security-related activity in Windows Event Viewer (EVTX) files.

Introduction evtx-hunter helps to quickly spot interesting security-related activity in Windows Event Viewer (EVTX) files. It can process a high numbe

NVISO 116 Dec 29, 2022
CVE-2022-23046 - SQL Injection Vulnerability on PhpIPAM v1.4.4

CVE-2022-23046 PhpIPAM v1.4.4 allows an authenticated admin user to inject SQL s

2 Feb 15, 2022
Threat Intel Platform for T-POTs

GreedyBear The project goal is to extract data of the attacks detected by a TPOT or a cluster of them and to generate some feeds that can be used to p

The Honeynet Project 72 Jan 01, 2023
EMBArk - The firmware security scanning environment

Embark is being developed to provide the firmware security analyzer emba as a containerized service and to ease accessibility to emba regardless of system and operating system.

emba 175 Dec 14, 2022
Rapidly enumerate subdomains and domains using rapiddns.io.

Description Simple python module (unofficial) allowing you to access data from rapiddns.io. You can also use it as a module. As mentioned on the rapid

27 Dec 31, 2022
⛤Keylogger Generator for Windows written in Python⛤

⛤Keylogger Generator for Windows written in Python⛤

FZGbzuw412 33 Nov 24, 2022
Driver Buddy Reloaded is an IDA Pro Python plugin that helps automate some tedious Windows Kernel Drivers reverse engineering tasks.

Driver Buddy Reloaded Quickstart Table of Contents Installation Usage About Driver Buddy Reloaded Finding DispatchDeviceControl Labelling WDM & WDF St

Paolo 'VoidSec' Stagno 199 Jan 04, 2023
TightVNC Vulnerability.

CVE-2022-23967 In TightVNC 1.3.10, there is an integer signedness error and resultant heap-based buffer overflow in InitialiseRFBConnection in rfbprot

MaherAzzouzi 15 Jul 11, 2022
A dynamic multi-STL, multi-process OpenSCAD build system with autoplating support

scad-build This is a multi-STL OpenSCAD build system based around GNU make. It supports dynamic build targets, intelligent previews with user-defined

Jordan Mulcahey 1 Dec 21, 2021
Subdomain enumeration,Web scraping and finding usernames automation script written in python

Subdomain enumeration,Web scraping and finding usernames automation script written in python

Syam 12 Nov 22, 2022
Automatically download all 10,000 CryptoPunk NFTs.

CryptoPunk Stealer The sole purpose of this script is to download the entire CryptoPunk NFT collection. How does it work? Basically, the website where

Dan 7 Oct 22, 2022
logmap: Log4j2 jndi injection fuzz tool

logmap - Log4j2 jndi injection fuzz tool Used for fuzzing to test whether there are log4j2 jndi injection vulnerabilities in header/body/path Use http

之乎者也 67 Oct 25, 2022
Password List Maker

Red-Key Red-Key Password List Maker Version 1.1.2 Created By FireKing255 -=Features=- Create Random Password List Create Password List Create Password

FireKing255 7 Dec 26, 2021
This is an advanced backdoor, created with Python

Backdoor This is a Backdoor, created with Python 3. Types of Commands: Downloading / Uploading files. Launching / Deleting / Reading file's content. S

swagkarna 28 Oct 28, 2022
OpenPort scanner GUI tool (CNMAP)

CNMAP-GUI- OpenPort scanner GUI tool (CNMAP) as you know it is the advanced tool to find open port, firewalls and we also added here heartbleed scanni

9 Mar 05, 2022
Raphael is a vulnerability scanning tool based on Python3.

Raphael Raphael是一款基于Python3开发的插件式漏洞扫描工具。 Raphael is a vulnerability scanning too

b4zinga 5 Mar 21, 2022
Fuzzercorn - Bring libfuzzer to Unicorn

Fuzzercorn libfuzzer bindings for Unicorn. API // The main entry point of the fu

lazymio 23 Nov 17, 2022
A hack for writing switch statements with type annotations in Python.

py_annotation_switch A hack for writing switch statements in type annotations for Python. Why should I use this? You most definitely should not use th

6 Oct 17, 2021
Moodle community-based vulnerability scanner

badmoodle Moodle community-based vulnerability scanner Description badmoodle is an unofficial community-based vulnerability scanner for moodle that sc

Michele Di Bonaventura 11 Dec 22, 2022