Dcf-game-infrastructure-public - Contains all the components necessary to run a DC finals (attack-defense CTF) game from OOO

Overview

dcf-game-infrastructure

All the components necessary to run a game of the OOO DC CTF finals.

Authors: adamd, hacopo, Erik Trickel, Zardus, and bboe

Design Philosophy

This repo contains all the game components necessary to run an Attack-Defense CTF that OOO used from 2018--2021.

The design is based on adamd's experience building the ictf-framework.

There are fundamental tenenats that we try to follow in the design of the system:

Spoke component model

The communication design of the components in the system (which you can kind of think of as micro-services) is a "spoke" model, where every component talks to the database (through a RESTish API), and no component directly talks to any other.

In this way, each component can be updated separately and can also be scaled independently using our k8s hosting.

This also made testing of each component easier, as the only dependence on a component is on the state of the database.

The only exception to this is the patchbot (the component that needs to test the patches submitted by the teams).

The database API puts the patchbot testing jobs into an RQ (Redis Queue), which all the patchbot workers pull jobs from.

Append-only database design

Fundamentally, a CTF database needs to calculate scores (that's essentially what the teams care about).

Prior design approaches that we've used would have a points or score column in the team table, and when they acquired or lost points, the app code would change this value.

However, many crazy things can happen during a CTF: recalculating scores or missed flags, even changing the scoring functions itself.

These can be difficult to handle depending on how the system is developed.

Therefore, we created a completely append-only database model, where no data in the DB is ever deleted or changed.

Even things like service status (the GOOD, OK, LOW, BAD that we used) is not a column in the services table. Every change of status would created a new StatusIndicator row, and the services would pull the latest version from this table.

Event model

Related to the append-only database design, everything in the database was represented by events.

The database would store all game events (in our game over the years was SLA_SCRIPT, FLAG_STOLEN, SET_FLAG, KOH_SCORE_FETCH, KOH_RANKING, PCAP_CREATED, PCAP_RELEASED, and STEALTH).

Then, the state of the game is based on these events.

An additional benefit is that these events could be shipped to the teams as part of the game_state.json.

Separate k8s clusters

How we ran this is with two k8s clusters: an admin cluster and a game cluster.

The admin cluster ran all of these components.

The game cluster ran all of the CTF challenges.

We used this design to do things like drop flags on the services. The flagbot used kubectl to drop a flag onto a service running in the other cluster.

This also allowed us to lock down the game cluster so that the vulnerable services couldn't make external requests, could be scaled separately, etc.

Install Requirements

This package is pip installable, and installs all dependencies. Do the following in a virtualenv:

$ pip install -e .

NOTE: If you want to connect to a mysql server (such as in prod or when deving against a mysql server), install the mysqlclient dependency like so:

$ pip install -e .[mysql]

Testing

Make sure the tests pass before you commit, and add new test cases in test for new features.

Note the database API now checks that the timezone is in UTC, so you'll need to specify that to run the tests:

$ TZ=UTC nosetests -v

Local Dev

If you're using tmux, I created a script local_dev.sh that will run a database-api, database-api frontend, team-interface backend, team-interface frontend, gamebot, and an ipython session with a database client created.

Just run the following

$ ./local_dev.sh

Deploy to prod

Build and -p push the image to production registry.

$ ./deploy.sh -p

Won't -r restart the running services, need to do:

$ ./deploy.sh -p -r

database-api

This has the tables for the database, a REST API to access it, and a python client to access the REST API.

See ooogame/database for details.

flagbot

Responsible for putting new flags into all the services for every game tick.

See ooogame/flagbot for details.

fresh-flagbot

Responsible for putting a new flags into a pod when it first comes up (from a team patching the service).

See ooogame/fresh_flagbot for details.

gamebot

Responsible for incrementing the game's ticks.

See ooogame/gamebot for details.

koh-scorebot

Responsible for extracting the King of the Hill (koh) scores from all the koh pods every tick, and submitting them to the database.

See ooogame/koh_scorebot for details.

team-interface

Responsible for providing an interface to the teams so that they can submit flags, get pcaps, upload patches, and get their patch status. Split into a backend flask REST API, which essentially wraps the database-api, and a React frontend.

See ooogame/team_interface for details.

pcapbot

Responsible for picking up all the newly generated pcaps, anonymize them, and if the service is releasing pcaps then release them.

See ooogame/pcapbot for details.

gamestatebot

Responsible for creating the game state at every new tick and storing them in the nfs, and release them publicly.

See ooogame/gamestatebot for details.

This is also the component that pushes data to the public scoreboard

Owner
Order of the Overflow
Order of the Overflow
We utilize deep reinforcement learning to obtain favorable trajectories for visual-inertial system calibration.

Unified Data Collection for Visual-Inertial Calibration via Deep Reinforcement Learning Update: The lastest code will be updated in this branch. Pleas

ETHZ ASL 27 Dec 29, 2022
Using Clinical Drug Representations for Improving Mortality and Length of Stay Predictions

Using Clinical Drug Representations for Improving Mortality and Length of Stay Predictions Usage Clone the code to local. https://github.com/tanlab/MI

Computational Biology and Machine Learning lab @ TOBB ETU 3 Oct 18, 2022
A Python library for Deep Graph Networks

PyDGN Wiki Description This is a Python library to easily experiment with Deep Graph Networks (DGNs). It provides automatic management of data splitti

Federico Errica 194 Dec 22, 2022
Implement some metaheuristics and cost functions

Metaheuristics This repot implement some metaheuristics and cost functions. Metaheuristics JAYA Implement Jaya optimizer without constraints. Cost fun

Adri1G 1 Mar 23, 2022
Metric learning algorithms in Python

metric-learn: Metric Learning in Python metric-learn contains efficient Python implementations of several popular supervised and weakly-supervised met

1.3k Dec 28, 2022
A flexible ML framework built to simplify medical image reconstruction and analysis experimentation.

meddlr Getting Started Meddlr is a config-driven ML framework built to simplify medical image reconstruction and analysis problems. Installation To av

Arjun Desai 36 Dec 16, 2022
For storing the complete exploration of Visual Question Answering for our B.Tech Project

Multi-Image vqa @authors: Akhilesh, Janhavi, Harsh Paper summary, Ideas tried and their corresponding results: on wiki Other discussions: on discussio

Harsh Raj 3 Jun 16, 2022
A public available dataset for road boundary detection in aerial images

Topo-boundary This is the official github repo of paper Topo-boundary: A Benchmark Dataset on Topological Road-boundary Detection Using Aerial Images

Zhenhua Xu 79 Jan 04, 2023
An official implementation of "Exploiting a Joint Embedding Space for Generalized Zero-Shot Semantic Segmentation" (ICCV 2021) in PyTorch.

Exploiting a Joint Embedding Space for Generalized Zero-Shot Semantic Segmentation This is an official implementation of the paper "Exploiting a Joint

CV Lab @ Yonsei University 35 Oct 26, 2022
Adversarial Reweighting for Partial Domain Adaptation

Adversarial Reweighting for Partial Domain Adaptation Code for paper "Xiang Gu, Xi Yu, Yan Yang, Jian Sun, Zongben Xu, Adversarial Reweighting for Par

12 Dec 01, 2022
Price-Prediction-For-a-Dream-Home - A machine learning based linear regression trained model for house price prediction.

Price-Prediction-For-a-Dream-Home ROADMAP TO THIS LINEAR REGRESSION BASED HOUSE PRICE PREDICTION PREDICTION MODEL Import all the dependencies of the p

DIKSHA DESWAL 1 Dec 29, 2021
Benchmark for evaluating open-ended generation

OpenMEVA Contributed by Jian Guan, Zhexin Zhang. Thank Jiaxin Wen for DeBugging. OpenMEVA is a benchmark for evaluating open-ended story generation me

25 Nov 15, 2022
A tensorflow=1.13 implementation of Deconvolutional Networks on Graph Data (NeurIPS 2021)

GDN A tensorflow=1.13 implementation of Deconvolutional Networks on Graph Data (NeurIPS 2021) Abstract In this paper, we consider an inverse problem i

4 Sep 13, 2022
Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)

Official implementation of GOCor This is the official implementation of our paper : GOCor: Bringing Globally Optimized Correspondence Volumes into You

Prune Truong 71 Nov 18, 2022
PyTorch implementation for COMPLETER: Incomplete Multi-view Clustering via Contrastive Prediction (CVPR 2021)

Completer: Incomplete Multi-view Clustering via Contrastive Prediction This repo contains the code and data of the following paper accepted by CVPR 20

XLearning Group 72 Dec 07, 2022
A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution

DRSAN A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution Karam Park, Jae Woong Soh, and Nam Ik Cho Environments U

4 May 10, 2022
An official implementation of MobileStyleGAN in PyTorch

MobileStyleGAN: A Lightweight Convolutional Neural Network for High-Fidelity Image Synthesis Official PyTorch Implementation The accompanying videos c

Sergei Belousov 602 Jan 07, 2023
[CVPR2021] Invertible Image Signal Processing

Invertible Image Signal Processing This repository includes official codes for "Invertible Image Signal Processing (CVPR2021)". Figure: Our framework

Yazhou XING 281 Dec 31, 2022
Microscopy Image Cytometry Toolkit

Cytokit Cytokit is a collection of tools for quantifying and analyzing properties of individual cells in large fluorescent microscopy datasets with a

Hammer Lab 106 Jan 06, 2023
Riemann Noise Injection With PyTorch

Riemann Noise Injection - PyTorch A module for modeling GAN noise injection based on Riemann geometry, as described in Ruili Feng, Deli Zhao, and Zhen

2 May 27, 2022