The Multi-Mission Maximum Likelihood framework (3ML)

Overview

CI Conda Build and Publish Test Against XSPEC codecov Documentation Status License

GitHub pull requests GitHub issues

PyPi

PyPI version fury.io PyPI - Downloads PyPI - Python Version

Conda

Conda Conda

drawing

The Multi-Mission Maximum Likelihood framework (3ML)

A framework for multi-wavelength/multi-messenger analysis for astronomy/astrophysics.


Astrophysical sources are observed by different instruments at different wavelengths with an unprecedented quality. Putting all these data together to form a coherent view, however, is a very difficult task. Indeed, each instrument and data type has its own ad-hoc software and handling procedure, which present steep learning curves and do not talk to each other.

The Multi-Mission Maximum Likelihood framework (3ML) provides a common high-level interface and model definition, which allows for an easy, coherent and intuitive modeling of sources using all the available data, no matter their origin. At the same time, thanks to its architecture based on plug-ins, 3ML uses under the hood the official software of each instrument, the only one certified and maintained by the collaboration which built the instrument itself. This guarantees that 3ML is always using the best possible methodology to deal with the data of each instrument.

drawing

Though Maximum Likelihood is in the name for historical reasons, 3ML is an interface to several Bayesian inference algorithms such as MCMC and nested sampling as well as likelihood optimization algorithms. Each approach to analysis can be seamlessly switched between allowing users to try different approaches quickly and without having to rewrite their model or data interfaces.

Like your XPSEC models? You can use them in 3ML as well as our growing selection of 1-,2- and 3-D models from our fast and customizable modeling language astromodels.

Installation

Installing with pip or conda is easy

pip install astromodels threeml
conda  install astromodels threeml -c threeml conda-forge 

Please refer to the Installation instructions for more details and trouble-shooting.

Press

Who is using 3ML?

Here is a highlight list of teams and their publications using 3ML.

A full list of publications using 3ML is here.

Citing

If you find this package useful in you analysis, or the code in your own custom data tools, please cite:

Vianello et al. (2015)

Acknowledgements

3ML makes use of the Spanish Virtual Observatory's Filter Profile service (http://svo2.cab.inta-csic.es/svo/theory/fps3/index.php?mode=browse&gname=NIRT).

If you use these profiles in your research, please consider citing them by using the following suggested sentence in your paper:

"This research has made use of the SVO Filter Profile Service (http://svo2.cab.inta-csic.es/theory/fps/) supported from the Spanish MINECO through grant AyA2014-55216"

and citing the following publications:

The SVO Filter Profile Service. Rodrigo, C., Solano, E., Bayo, A. http://ivoa.net/documents/Notes/SVOFPS/index.html The Filter Profile Service Access Protocol. Rodrigo, C., Solano, E. http://ivoa.net/documents/Notes/SVOFPSDAL/index.html

ThreeML is supported by National Science Foundation (NSF)

Comments
  • Use from astromodels Xspec import * error

    Use from astromodels Xspec import * error

    Hello! When I want to use xspec library to combine GBM and XRT to analyze energy spectrum, I try to use from astromodels Xspec import *has the following error messages. How did this happen?


    ImportError Traceback (most recent call last) /tmp/ipykernel_12482/3995023305.py in 5 6 # we will need XPSEC models for extinction ----> 7 from astromodels.xspec import * 8 from astromodels.xspec.xspec_settings import *

    ~/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/init.py in ----> 1 from .factory import * 2 #from .xspec_settings import x

    ~/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/factory.py in 16 pass 17 ---> 18 from astromodels.xspec import _xspec 19 20 # When running in a Anaconda environment, the package xspec-modelsonly

    ImportError: cannot import name '_xspec' from 'astromodels.xspec' (/home/hank/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/init.py)

    opened by ghost 64
  • How do I install a stable threeml module?

    How do I install a stable threeml module?

    Hello,

    The threeml module is currently latest, causing many modules to fail to install, including the Xspexcmodule. So I want to install a more stable threeml module, so how do I do that?

    opened by ghost 18
  • Background Modeling

    Background Modeling

    I've been looking at the background modeling and it may be better to approach it by adding a node onto PointSource:

    ps = PointSource(...,background_shape=Powerlaw() )
    

    Otherwise, we are just adding a function inside of the plugin and have to dig into private members to access the free parameters of that function.

    Is this too involved of an approach @giacomov ?

    opened by grburgess 18
  • properties of the spectra and choosing sources for SpectrumLike

    properties of the spectra and choosing sources for SpectrumLike

    would it make sense to assign some properties in the SpectrumLike plugin. it is quite useful when one wants to associate only some sources to a given spectrum, instead of summing all, as it is done now.

    opened by volodymyrss 17
  • XS_gabs model is not correctly imported

    XS_gabs model is not correctly imported

    I am not sure yet what is the cause, but the model takes only 0 or 1 values, no intermediate, independently of the strength. Position and width still work, but fit breaks, naturally, due to lack of sensitivity to strength. cyclabs e.g. works well (but I don't need it now).

    opened by volodymyrss 17
  • Installation with Fermitools

    Installation with Fermitools

    Discussed in https://github.com/threeML/threeML/discussions/576

    Originally posted by PepeJoseHU September 11, 2022 Hello, I plan to install 3ML for Fermi data with CONDA, but looks 3ML have dependence with fermitools. But when I install 'fermitools', it also make an environment with CONDA, so how to fix this problem or if there have way to make connection between different CONDA environments?

    Thanks!

    opened by grburgess 16
  • Response and GBM files

    Response and GBM files

    There seems to be an issue with the reading of GBM DRMs (similar to what I have seen with gammapy)

    
    /usr/local/Cellar/python/HEAD/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/threeML-0.3.2-py2.7.egg/threeML/plugins/ogip.pyc in _read_matrix(self, data, n_channels)
        750             for j in range(n_grp[i]):
        751 
    --> 752                 rsp[i, f_chan[i][j]: f_chan[i][j] + n_chan[i][j]] = matrix[i][m_start:m_start + n_chan[i][j]]
        753 
        754                 m_start += n_chan[i][j]
    
    ValueError: could not broadcast input array from shape (128) into shape (127)
    
    
    
    bug 
    opened by grburgess 16
  • Plotting

    Plotting

    I propose some updates to plotting routines...

    1. Added a function to joint_likelihood.py that will return 1D/2D likelihood profiles/contours for all parameters/combinations of parameters for convenience.

    2. Added an option to plot_point_source_spectra() to also plot extended source spectra (this will just plot whatever was set as the spectral shape, i.e. the 'spatially integrated' spectrum.

    3a. Moved convergence_plots() and plot_chains() from bayesian_analysis.py to analysis_results.py, so that the plots can be produced from analysis results loaded from disk. 3b. Changed convergence_plots() to use the same number of samples for the bootstrapping average as for the sliding window, I hope that was the original intent.

    opened by henrikef 15
  • added upper/lower limits

    added upper/lower limits

    Added three functions based on the equal_tail_intervals to get the X percent upper/lower limits.

    This makes things rather convenient for finding lower limits on cutoffs (or flux normalizations) after doing Bayesian analysis, although I suppose it can also be used in a frequentist analysis once the variates are computed.

    opened by cbrisboi 13
  • "Current minimum stored after fit and current do not correspond!" messages with all PyGMO algorithms

    I am getting "Current minimum stored after fit and current do not correspond!" messages with all PyGMO algorithms for a very simple model and data, which works with MINUIT.

    opened by volodymyrss 12
  • Alternative to root_numpy

    Alternative to root_numpy

    I was looking at updating hawc_hal to use uproot rather than the old ROOT interface for its .root file io and saw that a big part of the root_numpy dependency is carried over from 3ML via the dependency on threeML/io/cern_root_utils. I would generally prefer to stay consistent with file handling between 3ML and hawc_hal. Is there a preference for uproot vs the new ROOT interface? This is already an issue in hawc_hal, see https://github.com/threeML/hawc_hal/issues/42

    The code looks straightforward to update, but before I run off and implement something that is not preferred, I thought asking would be wise first.

    no-issue-activity 
    opened by cbrisboi 11
  • "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    Hi,

    I'm running into the following fitting error while trying to fit just Fermi-LAT data: "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    I'm primarily interested in using 3ML to create joint fits of Fermi-LAT data with X-ray (NuSTAR/NICER/XMM/etc) data. I have already downloaded and created intermediate (ltcube, expo maps, etc). To recreate my error: I create a data from the DataList like so:

    eventFile = os.path.join( 'photon_file.fits' ) ft2File = os.path.join('spacecraft_file.fits' ) expomap = os.path.join( 'bexpmap_00.fits' ) ltcube = os.path.join( 'ltcube_00.fits' )

    if is_plugin_available("FermiLATLike"):

    LAT = FermiLATLike( "LAT", eventFile, ft2File, ltcube, 'binned', binned_expo_map=expomap, source_maps=srcmap )
    #I'm performing binned analysis on my source and have created the binned exposure and src maps.
    

    data = DataList(LAT)

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    Now I define my spectral model:

    spectrum=Log_parabola() source1=PointSource(test_src,ra=ra,dec=dec,spectral_shape=spectrum)

    model=Model(source1)

    fluxUnit=1/(u.MeVu.cm**2u.s)

    model.test_src.spectrum.main.Log_parabola.alpha.value=-2.7 model.test_src.spectrum.main.Log_parabola.alpha.bounds=(-5.0,-0.5) model.test_src.spectrum.main.Log_parabola.alpha.fix= False

    model.test_src.spectrum.main.Log_parabola.piv.value= 2134*u.MeV model.test_src.spectrum.main.Log_parabola.piv.fix= True

    model.test_src.spectrum.main.Log_parabola.K.value=2.2e-6fluxUnit model.test_src.spectrum.main.Log_parabola.K.bounds=(1e-12fluxUnit,1e1*fluxUnit) model.test_src.spectrum.main.Log_parabola.K.fix= False

    model.test_src.spectrum.main.Log_parabola.beta.value=0.3 model.test_src.spectrum.main.Log_parabola.beta.bounds=(0.0,1.0) model.test_src.spectrum.main.Log_parabola.beta.fix= False

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    And start the likelihood analysis: jl = JointLikelihood(model, data) res = jl.fit()

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    At this point, I run into the aforementioned error: "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    When investigating the model, there don't seem to be any constraints on the normalization (but this may be a product of assigning it a unit?):

                                           value min_value max_value  \
    

    test_src.spectrum.main.Log_parabola.K 0.0 0.0 0.0

    Any help would be appreciated.

    opened by aplangrangian 0
  • Graphics bug - Kills kernel

    Graphics bug - Kills kernel

    Describe the bug Importing TriggerSelector from GtBurst AFTER importing threeML causes crashes when viewing lightcurves or other plots. Specifically, after importing threeML then TriggerSelector, if one downloads GBM data, creates a TimeSeriesBuilder, and then goes to view the lightcurve, a Python3.9 window pops up but is unresponsive, and after force quitting it, the kernel also dies.

    NOTE: This is for BUG reports and not questions about generic data analysis.

    To Reproduce

    Run these commands in jupyter notebook in sequential order:

    from threeML import *

    from GtBurst.TriggerSelector import TriggerSelector

    gbm_data = download_GBM_trigger_data("bn090926181",detectors=['n3'])

    ts1 = TimeSeriesBuilder.from_gbm_cspec_or_ctime('n3', cspec_or_ctime_file=gbm_data['n3']["cspec"], rsp_file=gbm_data['n3']["rsp"] )

    fig = ts1.view_lightcurve(-50,100);

    Expected behavior I expect to view a light curve in the notebook, but a python3.9 unresponsive window pops up, and the kernel dies. This is fixed when importing TriggerSelector first, and then importing threeML.

    Screenshots Screen Shot 2022-12-16 at 11 11 28 AM

    Desktop (please complete the following information):

    • OS: iOS
    • Montery version 12.6
    • Apple M1 Pro chip
    opened by nkirschner 1
  • Finishing CastorLike and Ability to Build flux points.

    Finishing CastorLike and Ability to Build flux points.

    Feature Request

    I have been working on cleaning up and speeding up the castorLike plugin which is designed to fit flux points that have been generated by likelihood profiles (ew). However, I never use this feature, so it would be good to have some help.

    Describe the solution you'd like

    1. Clean up plotting
    2. Documentation and examples of usage
    3. We should build a tool that can actually generate these profiles from any spectral fit. Is it possible from just the FitResults alone?

    @henrikef @giacomov @omodei

    You peeps are probably the experts here. @giacomov started it off, so he might have some directions. Perhaps Eric Charles could be helpful as well?

    help wanted plugin issue 
    opened by grburgess 0
  • reuse extended event class files too

    reuse extended event class files too

    I added a line to include the extended LAT files with a different name also to the list of files to be checked if they were previously downloaded already.

    opened by maklinger 1
  • asymmetric error in XY like

    asymmetric error in XY like

    I have spectral points which have asymmetric error and I am hoping to use XY Like to fit those spectral points. However, it seems like XY Like only supports symmetric errors. Is it possible to make XY Like support asymmetric error?

    opened by jasonfan1997 15
  • Value of BIC

    Value of BIC

    Discussed in https://github.com/threeML/threeML/discussions/585

    Originally posted by changxuezhao September 28, 2022 Dear Dr. Burgess, I have two questions for you about 3ML. First of all, the statistical (whether chi2 or pgstat or BIC) obtained using XSPEC will be smaller than 3ML, I think this is the reason for the likelihood, my question is why the likelihood of 3ML is larger than other tools(xspec,rmfit). Second, why can only the line graph of vFv be drawn without data points, and how to add data points to the vFv graph. Thanks and looking forward to your reply。

    no-issue-activity statistics 
    opened by grburgess 1
Releases(v0.1.0)
  • v0.1.0(Apr 19, 2016)

  • v0.0.5(Feb 6, 2015)

    All dependencies are now explicitly stated in the setup.py file, with the exception of ROOT (we might get rid of this dependency and get MINUIT from somewhere else)

    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Feb 6, 2015)

    The user can install this version by using pip ("pip install threeML").

    The dependencies (numpy, scipy, numexpr, numdifftools...) must be installed separately.

    Source code(tar.gz)
    Source code(zip)
Owner
The Multi-Mission Maximum Likelihood (3ML)
The Multi-Mission Maximum Likelihood (3ML)
A Human-in-the-Loop workflow for creating HD images from text

A Human-in-the-Loop? workflow for creating HD images from text DALL·E Flow is an interactive workflow for generating high-definition images from text

Jina AI 2.5k Jan 02, 2023
MANO hand model porting for the GraspIt simulator

Learning Joint Reconstruction of Hands and Manipulated Objects - ManoGrasp Porting the MANO hand model to GraspIt! simulator Yana Hasson, Gül Varol, D

Lucas Wohlhart 10 Feb 08, 2022
Nsdf: A mesh SDF with just some code we can directly paste into our raymarcher

nsdf Representing SDFs of arbitrary meshes has been a bit tricky so far. Express

Jan Ivanecky 5 Feb 18, 2022
Python port of R's Comprehensive Dynamic Time Warp algorithm package

Welcome to the dtw-python package Comprehensive implementation of Dynamic Time Warping algorithms. DTW is a family of algorithms which compute the loc

Dynamic Time Warping algorithms 154 Dec 26, 2022
PG2Net: Personalized and Group PreferenceGuided Network for Next Place Prediction

PG2Net PG2Net:Personalized and Group Preference Guided Network for Next Place Prediction Datasets Experiment results on two Foursquare check-in datase

Urban Mobility 5 Dec 20, 2022
A knowledge base construction engine for richly formatted data

Fonduer is a Python package and framework for building knowledge base construction (KBC) applications from richly formatted data. Note that Fonduer is

HazyResearch 386 Dec 05, 2022
Research shows Google collects 20x more data from Android than Apple collects from iOS. Block this non-consensual telemetry using pihole blocklists.

pihole-antitelemetry Research shows Google collects 20x more data from Android than Apple collects from iOS. Block both using these pihole lists. Proj

Adrian Edwards 290 Jan 09, 2023
Infrastructure as Code (IaC) for a self-hosted version of Gnosis Safe on AWS

Welcome to Yearn Gnosis Safe! Setting up your local environment Infrastructure Deploying Gnosis Safe Prerequisites 1. Create infrastructure for secret

Numan 16 Jul 18, 2022
AutoVideo: An Automated Video Action Recognition System

AutoVideo is a system for automated video analysis. It is developed based on D3M infrastructure, which describes machine learning with generic pipeline languages. Currently, it focuses on video actio

Data Analytics Lab at Texas A&M University 267 Dec 17, 2022
EMNLP 2021 - Frustratingly Simple Pretraining Alternatives to Masked Language Modeling

Frustratingly Simple Pretraining Alternatives to Masked Language Modeling This is the official implementation for "Frustratingly Simple Pretraining Al

Atsuki Yamaguchi 31 Nov 18, 2022
PyTorch version repo for CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes

Study-CSRNet-pytorch This is the PyTorch version repo for CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes

0 Mar 01, 2022
Home repository for the Regularized Greedy Forest (RGF) library. It includes original implementation from the paper and multithreaded one written in C++, along with various language-specific wrappers.

Regularized Greedy Forest Regularized Greedy Forest (RGF) is a tree ensemble machine learning method described in this paper. RGF can deliver better r

RGF-team 364 Dec 28, 2022
JAX code for the paper "Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation"

Optimal Model Design for Reinforcement Learning This repository contains JAX code for the paper Control-Oriented Model-Based Reinforcement Learning wi

Evgenii Nikishin 43 Sep 28, 2022
DAT4 - General Assembly's Data Science course in Washington, DC

DAT4 Course Repository Course materials for General Assembly's Data Science course in Washington, DC (12/15/14 - 3/16/15). Instructors: Sinan Ozdemir

Kevin Markham 779 Dec 25, 2022
Official implementation for paper Render In-between: Motion Guided Video Synthesis for Action Interpolation

Render In-between: Motion Guided Video Synthesis for Action Interpolation [Paper] [Supp] [arXiv] [4min Video] This is the official Pytorch implementat

8 Oct 27, 2022
A denoising diffusion probabilistic model (DDPM) tailored for conditional generation of protein distograms

Denoising Diffusion Probabilistic Model for Proteins Implementation of Denoising Diffusion Probabilistic Model in Pytorch. It is a new approach to gen

Phil Wang 108 Nov 23, 2022
This repository contains the code used in the paper "Prompt-Based Multi-Modal Image Segmentation".

Prompt-Based Multi-Modal Image Segmentation This repository contains the code used in the paper "Prompt-Based Multi-Modal Image Segmentation". The sys

Timo Lüddecke 305 Dec 30, 2022
Pretraining Representations For Data-Efficient Reinforcement Learning

Pretraining Representations For Data-Efficient Reinforcement Learning Max Schwarzer, Nitarshan Rajkumar, Michael Noukhovitch, Ankesh Anand, Laurent Ch

Mila 40 Dec 11, 2022
Pytorch implementation of paper Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data

Pytorch implementation of paper Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data

Hrishikesh Kamath 31 Nov 20, 2022
Cmsc11 arcade - Final Project for CMSC11

cmsc11_arcade Final Project for CMSC11 Developers: Limson, Mark Vincent Peñafiel

Gregory 1 Jan 18, 2022