Data portal client and server for NMDC.

Last update: Dec 14, 2022

Related tags

Overview

NMDC Server and Client Portal

Getting started with Docker

install ldc
install submodules via git submodule update --init --recursive

In order to populate the database, you must create a .env file in the top level directory containing mongo credentials.

# .env
NMDC_MONGO_HOST=changeme
NMDC_MONGO_USER=changeme
NMDC_MONGO_PASSWORD=changeme

With that file in place, populate the docker volume by running,

ldc run backend nmdc-server truncate # if necessary
ldc run backend nmdc-server migrate
ldc run backend nmdc-server ingest

Then you can start up the services.

ldc up

View main application at http://localhost:8080/ and the swagger page at http://localhost:8080/docs.

NOTE: If you have migration issues when starting up the server, you can purge the volume and start from scratch by running the following, then re-running ingestion (above).

ldc down
docker volume rm -f nmdc-server_app-db-data

Development Installation

Configuration

cp .env.example .env

Edit values in .env to point to existing postgresql databases.

OAuth setup

See nmdc_server/config for configuration. Env variable names begin with NMDC_.

At minimum, after creating a new OrcID API, you'll need to set these.

NMDC_CLIENT_ID=changeme
NMDC_CLIENT_SECRET=changeme
NMDC_HOST=http://localhost:8080

Run (development)

With docker and LDC

ldc dev up -d

With python virtualenv. Requires Python 3.7+

pip install -e .
pip install uvicorn tox

uvicorn nmdc_server.asgi:app --reload

View swagger page at http://localhost:8000/docs.

Testing

tox

Troubleshooting

Occasionally, a migration will fail to run correctly. This will cause data ingestion to fail. The fix is to drop the existing database and to restart the service. In docker-compose, this can be done by running

docker-compose run backend psql postgres -c 'drop database nmdc_a;'

On Spin, you can use the web interface to start a shell on the db instance and run:

psql -U postgres postgres -c 'drop database nmdc_a;'

Then, redeploy the backend service.

Generating new migrations

The way this project injects the database uri into alembic's configuration does not work when using alembic's CLI. To generate a new migration, you must modify nmdc_server/alembic.ini and hard code your database uri to sqlalchemy.url. After doing that, you can run alembic commands as usual, e.g.

alembic -c nmdc_server/alembic.ini revision --autogenerate

Developing with the shell

A handy IPython shell is provided with some commonly used symbols automatically imported, and autoreload 2 enabled. To run it:

ldc dev run --rm backend nmdc-server shell

You can also pass --print-sql to output all SQL queries.

Comments

Study page updates

GH issue for adding changes wanted on the study pages.

12/6/21: Would like the link to ESS-DIVE to be more prominent (logo?). Would like to have a mock-up to share with the ESS-DIVE team on January 24th.

opened by ssarrafan 19
Portal Download - bulk

Researchers who are interested in raw data generally want to do bulk downloads (to a server/cloud) for custom analyses. Researchers who are interested in data products may want to download to a server or to their local computer for custom analysis or sending to KBase. Researchers are also interested in downloading the associated sample metadata only.

Priority - medium Urgency - low
priority: medium X LARGE

opened by ssarrafan 18
Discussion of Hierarchical Search
I sat down to draft an API, and I couldn't shake the feeling that a bit more clarity was needed.

After giving this some more thought, I really can't get my head around the idea of combining Studies, Samples, and Omics Types into a single search interface. You end up needing tabs because there are too many concepts on the page at once.

I do not understand if this combined view is going to replace the existing "I am looking for..." pages.

I've noticed that there are omics types not associated with any sample, and I don't understand how that's possible. Here's an example. Neither OMC or Proteomics for that study have any samples. Is this an inconsistency?

is it possible to enable toggle between sample and study level here

^ this is a comment from miro. I believe that no matter how we end up doing this, the author of the comment is right: you have to differentiate between study, sample, and omics processing type as your first-order search result.

The fundamental problem

The problem we're trying to solve is not that different types of results appear on different tabs/pages, but that sometimes you want to filter one type of entity by another type of filter, and having to go from A to B, add a filter, then back to A to see results is too much of a burden.

For example,

filter studies by KO term (not really sure what entity KO term belongs to)

filter study by omics type (show me studies with any amount of Proteomics)

Filter samples by PI Name

In other words...

I think NMDC understands that, if they want to see studies (with some aggregations and vis) they go to the study page.

I think they are (rightfully) confused that, to see studies with Metabolomics processing on environmental samples, they have to visit 2 other pages and add filters first, then return to the study page.

Possible incremental solution

As we discussed earlier this week, I would actually be fine with doing this work in "new" routes. I'd like to have a simpler API, and the query string parameters could use simplifying. My goal here is to have a better idea of where this is going before I back myself into a corner.

To start with, we need to show all facets from Studies, Omics Types, and Samples on each page. Probably if you're on Studies, the study facets are at the top.

Study search updates

I think this is what the NMDC crew expected us to show up with. We were hand-waving at the "search by KO, filter by study" part that comes before sample search results are visible, but this was an important part of the query process to show.

A couple notes;

Flip the order of the aggregations and the search results. Aggregations go on top.

I stupidly only drew 1 row for study search results. That's a list.

Clicking on "40 samples" would take you to the sample page filtered by that study.

Clicking on "38 MegaG" would take you to the sample page filtered by that study AND only including samples with MetaG.

Another version I hacked together. Pretend it has a "samples" button for each study, and that it has KO search.

This is an incremental change. After this is done, we can have a discussion about whether or not it is actually necessary to embed a list of samples in a drop-down under studies, or if that suggestion came about because of the misunderstandings about filtering above.

Sample search updates

Will look exactly like it already does in figma. Maybe they group_by study_name for sort order, maybe we add dividers between groups, but otherwise, it's a paginated list of ~~studies~~ samples.
opened by subdavis 18
Portal Search - EnvO

This is a request to work on navigation for EnvO such that we can put something in front of users and iterate. EnvO has a high learning curve and the portal has potential to be integrated into training efforts if done well. What's in the portal now does not show relationships between EnvO terms and is not intuitive.

This will require working with Chris and Aim 1 (I do not expect Kitware to solve this on their own). Integrating search with EnvO support for aliases would be really powerful. Can we track the searches people are doing? (to try to understand EnvO vs. GOLD)

Priority - High Urgency - High
type: question priority: high

opened by ssarrafan 14
DOI broken in Bioscales study page

"Other Publications" Link looks odd to be pointing to OSTI for Bioscales. Also, clicking on the DOI takes you to an undefined DOI. @dehays can you help sort out what might be causing this?

type: bug

opened by pvangay 13
Design for portal landing pages

Requirements and feedback discussed in May, this issue is to work on the design and mock-ups incorporating the discussion and feedback.

Relates to https://github.com/microbiomedata/nmdc-server/issues/342 from May
type: feature

opened by ssarrafan 11
Portal Search - search space (Analysis, tech plan for July)

Right now you have to go into each search term on the left hand side and search within it. Would be helpful to be able to search across all terms with one search bar on top.

Priority - Low Urgency - Medium
priority: medium LARGE

opened by ssarrafan 11
Portal Search - KEGG

Enable search by modules and the 3 Pathway Levels. KO term is very granular. Oftentimes we're interested in understanding whether there's presence of an entire pathway (sometimes this may be specific, like "fatty acid metabolism" which is a level 3 pathway, but other times we may be interested in "metabolism" in general which is a level 1 pathway)

Priority - High Urgency - Medium
priority: high MEDIUM

opened by ssarrafan 10
Requirements for data landing pages

This issue is to track requirements for data landing pages. Please link to any related documentation, mock-ups, or diagrams developed. This sprint will not be focused on making these changes but just on the requirements.
type: needs discussion

opened by ssarrafan 10
Support basic "find and replace" feature

Search/Replace globally and cell by cell (iterate through and find/replace) would be ideal, advanced feature (nice to have): search/replace within selection.

Also helpful for search/replace of null values. Brandon: may be more complex to implement.

Potential to use google sheets API (to make use of their features)? Brandon to look into this from a technical feasibility? Some drawbacks include requiring google accounts.

opened by jeffbaumes 9
Fix metaT and metaP annotation search
Currently this search returns no matches on metaT in the bar chart:

We need to discover why metaT are not being populated and fix it so that this search returns 33 matches for metaT in the bar chart.

Some thoughts on the problem from @dehays and others:

There is a single mongo collection storing all annotations which should contain metaT, metaP, metaG annotations. Understand that table and how to distinguish between the annotation types.

Annotations should link back to omics_processing ids, which should have a definitive type (metaG, metaT, etc.).

metaT annotations are run separately from metaG, and there should be data parsed and populated in mongo.

nmdc_server does not seem to have a separate join path/table for metaT. Investigate why that is and how to add it.
opened by jeffbaumes 9
Ingest in transaction
This PR changes the ingestion to work inside of a transaction on a single database rather than via an A/B database.

Intro

Previously: We had two databases. The web server would operate on one database and ingestion would clear-then-load the other. Data that we persisted was copied from the live database to the other. We'd rotate the databases and re-deploy after a successful ingest.

With this PR: There is only one database. The ingestion opens a transaction and clears the database without locking reads. Changes from the ingestion are visible without intervention on the web server if ingestion succeeds. Changes are discarded if ingestion fails or disconnects.

Goals

The long-term goals of this change are:

Supporting the ability to persist the mga_gene_function table without a full ingest. Re-creating the mga_gene_function table takes 48+ hours. While this PR will still clear the mga_gene_function table without a full ingest, it lays the groundwork necessary to build a solution that persists this table without a full ingest. This would allow this table to be stale, but refresh other data (a partial ingest only takes ~1 hour). Work is needed to selectively drop rows in the mga_gene_function that violate FK constraints in the presence of altered dependent data.

Simplify operations. The A/B database is unconventional and difficult to reason about. This is particularly true in the presence of data that must be persisted which increases the risk of data loss.

Cons

This approach, however, has a con: database "bloat" is likely to accrue. With the A/B database, tables were truncated and therefore all indexes were rebuilt and no dead tuples remained after ingestion. Now, we DELETE FROM and we'll have dead tuples. Conventional tools can combat this. The scheduled VACUUM that's already active will take care of a lot of this. A VACUUM FULL will achieve the same results as the A/B approach, but does lock reads during it's process. There's pg_repack which can be used to VACUUM FULL without the lock.
opened by mcovalt 1
Consider increasing time until expiration for job results

Our task results are stored in redis, and results only stick around for 1 day. This can be frustrating for a job that can now take 48 hours +. This can be configured in our celery app. Increasing the amount of time we hold on to job results (e.g. for a week or forever) could be helpful for investigating jobs.

For example, if an ingest task fails, the details about that failure only persist for 1 day. If the error is not caught before then, it can be difficult to debug without retrying and checking the redis store every so often. If we hang on to the result longer, there is less of a chance we lose information in this manner.

opened by naglepuff 1
Hyperlink in descriptions sometimes broken

Issue: Some of the hyperlinks in the submission portal are broken. Or don't work with the pop up window. For example, env_broad_scale. Links when you double click on the header don't open. When you validate and move to "env_broad_scale" the side window has the link, but not the whole link works...... http://purl.obolibrary.org/obo/ENVO_00000428 ... everything after the _ is missing.

Resolution: All links work everywhere. In the side window and in the pop up The full link works.

opened by mslarae13 0
Submission Portal / Data Harmonizer: Upon validation, remain on current cell

Issue: Currently, when you validate a submission on data harmonizer the "active cell" is shifted to the first found invalid cell.

Request: Can we make it so it STAYS on the cell you're currently on? So if I'm checking geo_loc_name validation it stays there and doesn't move me to "unique ID" when that is invalid?

opened by mslarae13 0
Stegen biosample type mis-represented?
On the NMDC data portal, James Stegen's study "Riverbed sediment microbial communities from the Columbia River, Washington, USA" has all samples that start with GW-RW listed as sediment samples. https://data.microbiomedata.org/details/sample/gold:Gb0126430 https://data.microbiomedata.org/?q=ChgIABABGAMiECJnb2xkOkdzMDExNDY2MyI=

But, when you follow the Biosample sample identifier, it's a "groundwater metagenone" from a "sand" sample. img & gold say sediment

Biosample (groundwater metagenome, sand): https://www.ebi.ac.uk/biosamples/samples/SAMN06267115#:~:text=groundwater%20metagenome

Questions / resolution 1: Do we need to follow up on what is listed in biosample? 2: Is it our responsibility to make sure the external source is right? Do we assume GOLD and NMDC and IMG are right, and biosample is incorrectly represented?

If we assume we're correct, and we don't bother with updating biosample, this issue can be closed.

@emileyfadrosh , @aclum , @ssarrafan , FYI.
opened by mslarae13 0
Bump qs from 6.5.2 to 6.5.3 in /web
Bumps qs from 6.5.2 to 6.5.3.

Changelog

Sourced from qs's changelog.

6.5.3

[Fix] parse: ignore __proto__ keys (#428)

[Fix] utils.merge: avoid a crash with a null target and a truthy non-array source

[Fix] correctly parse nested arrays

[Fix] stringify: fix a crash with strictNullHandling and a custom filter/serializeDate (#279)

[Fix] utils: merge: fix crash when source is a truthy primitive & no options are provided

[Fix] when parseArrays is false, properly handle keys ending in []

[Fix] fix for an impossible situation: when the formatter is called with a non-string value

[Fix] utils.merge: avoid a crash with a null target and an array source

[Refactor] utils: reduce observable [[Get]]s

[Refactor] use cached Array.isArray

[Refactor] stringify: Avoid arr = arr.concat(...), push to the existing instance (#269)

[Refactor] parse: only need to reassign the var once

[Robustness] stringify: avoid relying on a global undefined (#427)

[readme] remove travis badge; add github actions/codecov badges; update URLs

[Docs] Clean up license text so it’s properly detected as BSD-3-Clause

[Docs] Clarify the need for "arrayLimit" option

[meta] fix README.md (#399)

[meta] add FUNDING.yml

[actions] backport actions from main

[Tests] always use String(x) over x.toString()

[Tests] remove nonexistent tape option

[Dev Deps] backport from main

Commits

298bfa5 v6.5.3

ed0f5dc [Fix] parse: ignore __proto__ keys (#428)

691e739 [Robustness] stringify: avoid relying on a global undefined (#427)

1072d57 [readme] remove travis badge; add github actions/codecov badges; update URLs

12ac1c4 [meta] fix README.md (#399)

0338716 [actions] backport actions from main

5639c20 Clean up license text so it’s properly detected as BSD-3-Clause

51b8a0b add FUNDING.yml

45f6759 [Fix] fix for an impossible situation: when the formatter is called with a no...

f814a7f [Dev Deps] backport from main

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies javascript
opened by dependabot[bot] 0

Releases(0.6.2)

0.6.2(Aug 31, 2022)
What's Changed

Upgrade schema and DataHArmonizer date (https://github.com/microbiomedata/nmdc-server/pull/776)

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.6.1...0.6.2
Source code(tar.gz)
Source code(zip)
0.6.1(Aug 30, 2022)
What's Changed

Editable date (https://github.com/microbiomedata/nmdc-server/pull/774)

Swapped DataHarmonizer columns (https://github.com/microbiomedata/nmdc-server/pull/773)

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.6.0...0.6.1
Source code(tar.gz)
Source code(zip)
0.6.0(Aug 16, 2022)
What's Changed

Auto-submit (https://github.com/microbiomedata/nmdc-server/pull/764)

Ingest fixes (https://github.com/microbiomedata/nmdc-server/pull/768)

UI fixes (https://github.com/microbiomedata/nmdc-server/pull/769)

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.5.7...0.6.0
Source code(tar.gz)
Source code(zip)
0.5.7(Jul 28, 2022)
What's Changed

Prevent any over-scroll behavior on data portal (https://github.com/microbiomedata/nmdc-server/pull/754)

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.5.6...0.5.7
Source code(tar.gz)
Source code(zip)
0.5.6(Jul 15, 2022)
What's Changed

Show friendly environmental terms in sample detail page

Show geographic location (city/state) in sample detail page

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.5.5...0.5.6
Source code(tar.gz)
Source code(zip)
0.5.5(Jul 13, 2022)
What's Changed

Add peatland [ENVO:00000044] to the select list for "local environmental context" in soil-based templates

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.5.4...0.5.5
Source code(tar.gz)
Source code(zip)
0.5.4(Jul 13, 2022)
What's Changed

A fix for ingestion by @mcovalt in https://github.com/microbiomedata/nmdc-server/pull/743

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.5.3...0.5.4
Source code(tar.gz)
Source code(zip)
0.5.3(Jul 5, 2022)
What's Changed

Submission pagination by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/734

User table by @DeepikaGhodki in https://github.com/microbiomedata/nmdc-server/pull/733

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.5.2...0.5.3
Source code(tar.gz)
Source code(zip)
0.5.2(Jun 27, 2022)
What's Changed

Upgrade nmdc-schema to v5.0.3 by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/725

Enabled navigation to study page on click by @DeepikaGhodki in https://github.com/microbiomedata/nmdc-server/pull/724

Horizontal scroll by @DeepikaGhodki in https://github.com/microbiomedata/nmdc-server/pull/727

Remove bioscales by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/729

Client/update dh to main by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/731

Fix nginx build by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/732

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.5.1...0.5.2
Source code(tar.gz)
Source code(zip)
0.5.1(Jun 17, 2022)
What's Changed

Fix matching order by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/721

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.5.0...0.5.1
Source code(tar.gz)
Source code(zip)
0.5.0(Jun 16, 2022)
What's Changed

Update App.vue by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/666

Update README docs by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/691

Embedded DataHarmonizer by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/667

Use node lts by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/695

Fix base url by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/696

Error improvements by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/698

Bump eventsource from 1.1.0 to 1.1.1 in /web by @dependabot in https://github.com/microbiomedata/nmdc-server/pull/688

Update .env.example by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/697

Referece page fix by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/699

Bump async from 2.6.3 to 2.6.4 in /web by @dependabot in https://github.com/microbiomedata/nmdc-server/pull/660

Bump protobufjs from 6.11.2 to 6.11.3 in /web by @dependabot in https://github.com/microbiomedata/nmdc-server/pull/693

Sidebar in DH Submission Portal by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/700

Submission home fix by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/701

Collapsed errors by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/703

Docs Improvements by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/704

Docs updates migrations by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/705

DB Import documentation. by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/706

Add user models by @DeepikaGhodki in https://github.com/microbiomedata/nmdc-server/pull/702

Fix migration by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/709

Fix user migrations ingest by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/710

Reorganize submission portal by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/712

Apply DH field settings to support conditional dropdowns by @pkalita-lbl in https://github.com/microbiomedata/nmdc-server/pull/707

Add pydantic schemas for metadata submission by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/713

Better error handling for record loading by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/714

New Contributors

@DeepikaGhodki made their first contribution in https://github.com/microbiomedata/nmdc-server/pull/702

@pkalita-lbl made their first contribution in https://github.com/microbiomedata/nmdc-server/pull/707

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.4.4...0.5.0
Source code(tar.gz)
Source code(zip)
0.4.4(Apr 28, 2022)
What's Changed

Fix Principal Investigator Orcid discovery by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/663

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.4.3...0.4.4
Source code(tar.gz)
Source code(zip)
0.4.3(Apr 28, 2022)
What's Changed

Revert mongo timeout. by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/662

Switch DataHarmonizer to prod by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/664

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.4.2...0.4.3
Source code(tar.gz)
Source code(zip)
0.4.2(Apr 27, 2022)
What's Changed

Fix Contributor role URL by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/640

Submission Portal Load and Save by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/641

LinkML Support by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/645

Module Search Support by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/646

Fix #563 by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/647

Changes for 4.19 by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/649

Ingest updates for timeouts by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/651

Fix ingest related to depth and depth2 by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/652

Updates for submission portal: by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/654

hrc -> hcr by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/656

Conform CreditAssociation to schema by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/659

Change links by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/661

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.4.1...0.4.2
Source code(tar.gz)
Source code(zip)
0.4.1(Mar 31, 2022)
What's Changed

Allow missing kegg description text by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/624

Migrate submission, add docs by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/629

Update auth.py by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/631

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.4.0...0.4.1
Source code(tar.gz)
Source code(zip)
0.4.0(Feb 25, 2022)

Source code(tar.gz)
Source code(zip)
0.3.1(Jan 26, 2022)
What's Changed

Bugfix support bulk emsl by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/587

Update schema by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/588

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.3.0...0.3.1
Source code(tar.gz)
Source code(zip)
0.3.0(Jan 24, 2022)
What's Changed

Submission Portal | Descriptions in fields :---------------:|:--------------: |

New features

Submission Portal MVP by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/574

Display CredIT Roles for contributors @subdavis in https://github.com/microbiomedata/nmdc-server/pull/586

Hyperlink description URLs @subdavis in https://github.com/microbiomedata/nmdc-server/pull/586

Bug fixes

Fixes #570 -- some data objects were miscatagorized by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/573

Add descriptions from schema to search vields by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/580

Handle unknown data object types by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/583

Fix map centered rendering @subdavis in https://github.com/microbiomedata/nmdc-server/pull/586

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.2.0...0.3.0
Source code(tar.gz)
Source code(zip)
0.2.0(Dec 29, 2021)
What's Changed

ENVO Tree | Text Search :---------------:|:--------------: |

Fix missing aggregation key error by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/466

Address some issues: by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/468

Fix CI that broke with FastAPI release by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/469

Refactor database and test code to remove global state by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/467

Add myself as admin user by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/470

Envo tree by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/471

Fix migration error by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/473

Improve response format of envo tree by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/475

Prune Envo tree by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/476

Infer per-facet envo tree roots at ingest time by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/478

Envo tree UI by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/479

Change celery reference by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/480

Generate envo trees independently per facet by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/481

Simplify envo tree building logic by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/482

Envo tree counts by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/484

Fix scroll glitch by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/485

Basic ipython shell helper by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/483

Bump url-parse from 1.5.1 to 1.5.3 in /web by @dependabot in https://github.com/microbiomedata/nmdc-server/pull/489

Ingest alternative ids by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/492

Switch to new Envo data source by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/491

Full text facet value search by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/493

Remove constraing again by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/494

Remove studies column by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/495

Order results descending by count by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/496

Prune internal envo tree nodes statically by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/497

Prune useless root nodes in Envo trees by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/498

Refactor/remove v1 by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/499

Update BiosampleVisGroup.vue tooltip by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/503

Protobuf query history by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/507

Bump axios from 0.21.1 to 0.21.2 in /web by @dependabot in https://github.com/microbiomedata/nmdc-server/pull/508

Persist state through login, allow bulk download without login by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/509

Bump path-parse from 1.0.6 to 1.0.7 in /web by @dependabot in https://github.com/microbiomedata/nmdc-server/pull/487

Envo bug by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/512

Upset click by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/513

text should say update by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/514

Include definitions from schema by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/516

Fix build by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/517

Allow resolving KEGG pathways and modules to KO term sets by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/518

Kegg text search by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/519

Kegg search by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/520

Add database connection pool tuning parameters by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/521

KEGG text search by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/523

Fix CLI migrate invocation by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/524

Remove ingest db by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/522

Fix CLI commands to use ingest database by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/527

Remove regex timeout by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/526

revert by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/529

Fix toolbar links by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/531

Optimize MGA ingest by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/530

Add brite links by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/532

Bump max WAL size from 1GB (default) to 2GB by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/533

More database tuning by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/534

Schema changes, Study Page Redesign by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/535

Add ESS dive column by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/536

October release updates by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/538

Multiomics special handling by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/539

MAGS nullable fields by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/541

Disable proxy buffering by @zachmullen in https://github.com/microbiomedata/nmdc-server/pull/542

ESS DIVE by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/543

Fix httpx dependency issue by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/549

Fix ingest download stats by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/561

Client/december minor updates by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/565

Dependency upgrades by @subdavis in https://github.com/microbiomedata/nmdc-server/pull/567

Full Changelog: https://github.com/microbiomedata/nmdc-server/compare/0.1.0...0.2.0
Source code(tar.gz)
Source code(zip)
0.1.0(Jul 22, 2021)

Source code(tar.gz)
Source code(zip)

Owner

National Microbiome Data Collaborative

GitHub Repository https://data.microbiomedata.org

Modular Python-based Twitch bot optimized for customizability and ease of use.

rasbot Modular Python-based Twitch bot optimized for customizability and ease of use. rasbot is a Python-based Twitch bot that runs on your Twitch acc

9 Dec 14, 2022

A Python wrapper around the Twitter API.

Python Twitter A Python wrapper around the Twitter API. By the Python-Twitter Developers Introduction This library provides a pure Python interface fo

3.4k Jan 01, 2023

An inline Telegram bot to keep your private messages hidden from prying eyes.

Hide This Bot Hide This Bot is an inline Telegram bot to keep your private messages hidden from prying eyes. How do I host it? Here is a brief gui

41 Dec 02, 2022

This python cheat utilizes PyMeow, PyMem, and others to enhance your CS:GO experience ;).

CSGO-Python-Cheat This python cheat utilizes PyMeow, PyMem, and others to enhance your CS:GO experience ;). Features Esp Tracers Chams (More to come)

1 Nov 30, 2021

Freqtrade 3commas wrapper for python

Freqtrade 3commas wrapper The aim of this project is to provide an easy way to integrate freqtrade with 3commas. The main reason someone would want to

43 Dec 17, 2022

PTV is a useful widget for trading view for doing paper trading when bar reply is enabled

PTV is a useful widget for trading view for doing paper trading when bar reply is enabled.(this feature did not implement in trading view)

39 Dec 26, 2022

This is a script to export logs from AWS CloudTrail to a local file.

cloudtrail-export-logs This is a script to export logs from AWS CloudTrail to a local file. Getting Started Prerequisites python 3 boto3 pip Installin

2 Jan 02, 2022

Bot inspirado no Baidu Antivírus

Baidu Bot Bot inspirado no lendário Baidu Antivírus Informações O programa foi inteiramente feito em Python, sinta-se livre para fazer qualquer altera

1 Dec 18, 2021

Fast IP address lookup

ipscoop Fast IP Scoop Table of Contents Installation CLI Getting Started Ref Installation To install ipscoop, simply: $ python3 -m pip install -U git+

6 Mar 16, 2022

Nyon-stream - A python script that uses webtorrent to stream nyaa videos directly to mpv

nyon-stream A rather shitty script that uses webtorrent to stream nyaa videos di

18 Feb 08, 2022

The EscapePod Python SDK for Cyb3rVector's EscapePod Extension Proxy

EscapePod Extension SDK for Python by cyb3rdog This is the EscapePod Python SDK for Cyb3rVector's EscapePod Extension Proxy. With this SDK, you can: m

3 Mar 07, 2022

Telegram Group Calls Streaming bot with some useful features, written in Python with Pyrogram and Py-Tgcalls. Supporting platforms like Youtube, Spotify, Resso, AppleMusic, Soundcloud and M3u8 Links.

Yukki Music Bot Yukki Music Bot is a Powerful Telegram Music+Video Bot written in Python using Pyrogram and Py-Tgcalls by which you can stream songs,

996 Dec 28, 2022

Data portal client and server for NMDC.

Related tags

Overview

NMDC Server and Client Portal

Getting started with Docker

Development Installation

Configuration

OAuth setup

Run (development)

Testing

Troubleshooting

Generating new migrations

Developing with the shell

Comments

The fundamental problem

Possible incremental solution

Study search updates

Sample search updates

Intro

Goals

Cons

6.5.3

Releases(0.6.2)

0.6.2(Aug 31, 2022)

What's Changed

0.6.1(Aug 30, 2022)

What's Changed

0.6.0(Aug 16, 2022)

What's Changed

0.5.7(Jul 28, 2022)

What's Changed

0.5.6(Jul 15, 2022)

What's Changed

0.5.5(Jul 13, 2022)

What's Changed

0.5.4(Jul 13, 2022)

What's Changed

0.5.3(Jul 5, 2022)

What's Changed

0.5.2(Jun 27, 2022)

What's Changed

0.5.1(Jun 17, 2022)

What's Changed

0.5.0(Jun 16, 2022)

What's Changed

New Contributors

0.4.4(Apr 28, 2022)

What's Changed

0.4.3(Apr 28, 2022)

What's Changed

0.4.2(Apr 27, 2022)

What's Changed

0.4.1(Mar 31, 2022)

What's Changed

0.4.0(Feb 25, 2022)

0.3.1(Jan 26, 2022)

What's Changed

0.3.0(Jan 24, 2022)

What's Changed

New features

Bug fixes

0.2.0(Dec 29, 2021)

What's Changed

0.1.0(Jul 22, 2021)

Owner

National Microbiome Data Collaborative

Modular Python-based Twitch bot optimized for customizability and ease of use.

A Python wrapper around the Twitter API.

An inline Telegram bot to keep your private messages hidden from prying eyes.

This python cheat utilizes PyMeow, PyMem, and others to enhance your CS:GO experience ;).

Freqtrade 3commas wrapper for python

PTV is a useful widget for trading view for doing paper trading when bar reply is enabled

This is a script to export logs from AWS CloudTrail to a local file.

Bot inspirado no Baidu Antivírus

Fast IP address lookup

Nyon-stream - A python script that uses webtorrent to stream nyaa videos directly to mpv

The EscapePod Python SDK for Cyb3rVector's EscapePod Extension Proxy

Telegram Group Calls Streaming bot with some useful features, written in Python with Pyrogram and Py-Tgcalls. Supporting platforms like Youtube, Spotify, Resso, AppleMusic, Soundcloud and M3u8 Links.

HelpDESK Dynamics

Yes, it's true :orange_heart: This repository has 346 stars.