Learning nonlinear operators via DeepONet

Related tags

Deep Learningdeeponet
Overview

DeepONet: Learning nonlinear operators

DOI

The source code for the paper Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Nature Machine Intelligence, 2021.

System requirements

Most code is written in Python 3, and depends on the deep learning package DeepXDE. Some code is written in Matlab (version R2019a).

Installation guide

  1. Install Python 3
  2. Install DeepXDE (https://github.com/lululxvi/deepxde)
  3. Optional: For CNN, install Matlab and TensorFlow 1; for Seq2Seq, install PyTorch

The installation may take between 10 minutes and one hour.

Demo

Case Antiderivative

  1. Open deeponet_pde.py, and choose the parameters/setup in the functions main() and ode_system() based on the comments;
  2. Run deeponet_pde.py, which will first generate the two datasets (training and test) and then train a DeepONet. The training and test MSE errors will be displayed in the screen.

A standard output is

Building operator neural network...
'build' took 0.104784 s

Generating operator data...
'gen_operator_data' took 20.495655 s

Generating operator data...
'gen_operator_data' took 168.944620 s

Compiling model...
'compile' took 0.265885 s

Initializing variables...
Training model...

Step      Train loss    Test loss     Test metric
0         [1.09e+00]    [1.11e+00]    [1.06e+00]
1000      [2.57e-04]    [2.87e-04]    [2.76e-04]
2000      [8.37e-05]    [9.99e-05]    [9.62e-05]
...
50000     [9.98e-07]    [1.39e-06]    [1.09e-06]

Best model at step 46000:
  train loss: 6.30e-07
  test loss: 9.79e-07
  test metric: [7.01e-07]

'train' took 324.343075 s

Saving loss history to loss.dat ...
Saving training data to train.dat ...
Saving test data to test.dat ...
Restoring model from model/model.ckpt-46000 ...

Predicting...
'predict' took 0.056257 s

Predicting...
'predict' took 0.012670 s

Test MSE: 9.269857471315847e-07
Test MSE w/o outliers: 6.972881784590493e-07

You can get the training and test errors in the end of the output.

The run time could be between several minutes to several hours depending on the parameters you choose, e.g., the dataset size and the number of iterations for training.

Case Stochastic ODE/PDE

  1. Open sde.py, and choose the parameters/setup in the functions main();
  2. Run sde.py, which will generate traning and test datasets;
  3. Open deeponet_dataset.py, and choose the parameters/setup in the functions main();
  4. Run deeponet_dataset.py to train a DeepONet. The training and test MSE errors will be displayed in the screen.

Case 1D Caputo fractional derivative

  1. Go to the folder fractional;
  2. Run Caputo1D.m to generate training and test datasets. One can specify the orthongonal polynomial to be Legendre polynomial or poly-fractonomial in Orthogonal_polynomials.m. Expected run time: 20 mins.
  3. Run datasets.py to pack and compress the genrated datasets. Expected outputs: compressed .npz files. Expected run time: 5 mins.
  4. Run DeepONet_float32_batch.py to train and test DeepONets. Expected outputs: a figure of training and test losses. Expected run time: 1 hour.

Case 2D fractional Laplacian

Learning a 2D fractional Laplacian using DeepONets

  1. Run Fractional_Lap_2D.m to generate training and test datasets. Expected outputs: text files that store the training and test data. Expected run time: 40 mins.
  2. Run datasets.py to pack and compress the genrated datasets. Expected outputs: compressed .npz files. Expected run time: 15 mins.
  3. Run DeepONet_float32_batch.py to train and test DeepONets. Expected run time: 3 hours.

Learning a 2D fractional Laplacian using CNNs

  1. Suppose that the text files containing all training and test sets have been generated in the previous step.
  2. Run CNN_operator_alpha.py to train and test CNNs. Expected outputs: a figure of training and test losses. Expected run time: 30 mins.

Seq2Seq

  1. Open seq2seq_main.py, choose the problem in the function main(), and change the parameters/setup in the corresponding function (antiderivative()/pendulum()) if needed.
  2. Run seq2seq_main.py, which will first generate the dataset and then train the Seq2Seq model on the dataset. The training and test MSE errors will be displayed in the screen. Moreover, the loss history, generated data and trained best model will be saved in the direction ('./outputs/').

A standard output is

Training...
0             Train loss: 0.21926558017730713         Test loss: 0.22550159692764282
1000       Train loss: 0.0022761737927794456     Test loss: 0.0024939212016761303
2000       Train loss: 0.0004760705924127251     Test loss: 0.0005566366016864777
...
49000     Train loss: 1.2885914202342974e-06    Test loss: 1.999963387788739e-06
50000     Train loss: 1.1382834372852813e-06    Test loss: 1.8525416862757993e-06
Done!
'run' took 747.5421471595764 s
Best model at iteration 50000:
Train loss: 1.1382834372852813e-06 Test loss: 1.8525416862757993e-06

You can get the training and test errors in the end of the output.

The run time could be between several minutes to several hours depending on the parameters you choose, e.g., the dataset size and the number of iterations for training.

Instructions for use

The instructions for running each case are as follows.

  • Legendre transform: The same as Antiderivative in Demo. You need to modify the function main() in deeponet_pde.py.
  • Antiderivative: In Demo.
  • Fractional (1D): In Demo.
  • Fractional (2D): In Demo.
  • Nonlinear ODE: The same as Antiderivative in Demo. You need to modify the functions main() and ode_system() in deeponet_pde.py.
  • Gravity pendulum: The same as Antiderivative in Demo. You need to modify the functions main() and ode_system() in deeponet_pde.py.
  • Diffusion-reaction: The same as Antiderivative in Demo. You need to modify the function main() in deeponet_pde.py.
  • Advection: The same as Antiderivative in Demo. You need to modify the functions main() in deeponet_pde.py, run() in deeponet_pde.py, CVCSystem() in system.py, and solve_CVC() in CVC_solver.py to run each case.
  • Advection-diffusion: The same as Antiderivative in Demo. You need to modify the function main() in deeponet_pde.py.
  • Stochastic ODE/PDE: In Demo.

Questions

To get help on how to use the data or code, simply open an issue in the GitHub "Issues" section.

Comments
  • More than one input function maps to more than one output function by DeepONet

    More than one input function maps to more than one output function by DeepONet

    Hello Dr. Lu, It is very nice to get your reply. Following your answer, I think the data of the branch nets may be like this: [ u1(x1), ..., u1(xm) ] for one input function u1; [ u1(x1), ..., u1(xm), u2(x1), ..., u2(xm) ] for two input function u1 & u2. Or like this: [ u1(x1), u2(x1),..., u1(xm), u2(xm) ] ? I think the former seems to be the case. Do I have the correct understanding? I am looking forward to your reply. Thanks in advance.

    Originally posted by @WangYicunZJU in https://github.com/lululxvi/deeponet/issues/9#issuecomment-901184441

    opened by LVJIAQI777 5
  • Tests not running properly

    Tests not running properly

    Hi @lululxvi , Thanks for putting together deepxde. I was trying to get started with ODESystem but noticed a couple of things that are off. First off, here is my version info:

    In [1]: import tensorflow, deepxde
    
    In [2]: tensorflow.__version__
    Out[2]: '2.6.0'
    
    In [3]: deepxde.__version__
    Out[3]: '0.13.5'
    
    

    Now if I try to run the gravity pendulum ODE, I get a weird type mismatch error:

    ---------------------------------------------------------------------------
    TypeError                                 Traceback (most recent call last)
    <ipython-input-6-359904db8a4e> in <module>()
         47 
         48 if __name__ == "__main__":
    ---> 49     main()
    
    14 frames
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
        992           except Exception as e:  # pylint:disable=broad-except
        993             if hasattr(e, "ag_error_metadata"):
    --> 994               raise e.ag_error_metadata.to_exception(e)
        995             else:
        996               raise
    
    TypeError: in user code:
    
        /usr/local/lib/python3.7/dist-packages/deepxde/model.py:169 outputs_losses  *
            losses = self.data.losses(targets, outputs_, loss_fn, self)
        /usr/local/lib/python3.7/dist-packages/deepxde/data/triple.py:32 losses  *
            return [loss(targets, outputs)]
        /usr/local/lib/python3.7/dist-packages/deepxde/losses.py:27 mean_squared_error  *
            return bkd.reduce_mean(bkd.square(y_true - y_pred))
        /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_ops.py:1383 binary_op_wrapper
            raise e
        /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_ops.py:1367 binary_op_wrapper
            return func(x, y, name=name)
        /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:206 wrapper
            return target(*args, **kwargs)
        /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_ops.py:548 subtract
            return gen_math_ops.sub(x, y, name)
        /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_math_ops.py:10654 sub
            "Sub", x=x, y=y, name=name)
        /usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/op_def_library.py:558 _apply_op_helper
            inferred_from[input_arg.type_attr]))
    
        TypeError: Input 'y' of 'Sub' Op has type float32 that does not match type float64 of argument 'x'.
    
    

    Here is the piece of code that I am trying to run (I cloned and manually appended the path to the git repo to directly import some of the submodules)

    from __future__ import absolute_import
    from __future__ import division
    from __future__ import print_function
    
    import itertools
    
    import numpy as np
    import sys, os
    
    sys.path.append("deeponet/src/")
    
    import deepxde as dde
    from spaces import FinitePowerSeries, FiniteChebyshev, GRF
    from system import LTSystem, ODESystem, DRSystem, CVCSystem, ADVDSystem
    from utils import merge_values, trim_to_65535, mean_squared_error_outlier, safe_test
    
    def test_u_ode(nn, system, T, m, model, data, u, fname, num=100):
        """Test ODE"""
        sensors = np.linspace(0, T, num=m)[:, None]
        sensor_values = u(sensors)
        x = np.linspace(0, T, num=num)[:, None]
        X_test = [np.tile(sensor_values.T, (num, 1)), x]
        y_test = system.eval_s_func(u, x)
        if nn != "opnn":
            X_test = merge_values(X_test)
        y_pred = model.predict(data.transform_inputs(X_test))
        np.savetxt(fname, np.hstack((x, y_test, y_pred)))
        print("L2relative error:", dde.metrics.l2_relative_error(y_test, y_pred))
    
    def ode_system(T):
        """ODE"""
    
        def g(s, u, x):
            # Gravity pendulum
            k = 1
            return [s[1], -k * np.sin(s[0]) + u]
    
        s0 = [0, 0]  # Gravity pendulum
        return ODESystem(g, s0, T)
    
    def run(problem, system, space, T, m, nn, net, lr, epochs, num_train, num_test):
        # space_test = GRF(1, length_scale=0.1, N=1000, interp="cubic")
    
        X_train, y_train = system.gen_operator_data(space, m, num_train)
        X_test, y_test = system.gen_operator_data(space, m, num_test)
        if nn != "opnn":
            X_train = merge_values(X_train)
            X_test = merge_values(X_test)
    
        # np.savez_compressed("train.npz", X_train0=X_train[0], X_train1=X_train[1], y_train=y_train)
        # np.savez_compressed("test.npz", X_test0=X_test[0], X_test1=X_test[1], y_test=y_test)
        # return
    
        # d = np.load("train.npz")
        # X_train, y_train = (d["X_train0"], d["X_train1"]), d["y_train"]
        # d = np.load("test.npz")
        # X_test, y_test = (d["X_test0"], d["X_test1"]), d["y_test"]
    
        X_test_trim = trim_to_65535(X_test)[0]
        y_test_trim = trim_to_65535(y_test)[0]
        if nn == "opnn":
            data = dde.data.Triple(
                X_train=X_train, y_train=y_train, X_test=X_test_trim, y_test=y_test_trim
            )
        else:
            data = dde.data.DataSet(
                X_train=X_train, y_train=y_train, X_test=X_test_trim, y_test=y_test_trim
            )
    
        model = dde.Model(data, net)
        model.compile("adam", lr=lr, metrics=[mean_squared_error_outlier])
        checker = dde.callbacks.ModelCheckpoint(
            "model/model.ckpt", save_better_only=True, period=1000
        )
        losshistory, train_state = model.train(epochs=epochs, callbacks=[checker])
        print(
            "# Parameters:",
            np.sum([np.prod(v.get_shape().as_list()) for v in tf.trainable_variables()]),
        )
        dde.saveplot(losshistory, train_state, issave=True, isplot=True)
    
        model.restore("model/model.ckpt-" + str(train_state.best_step), verbose=1)
        safe_test(model, data, X_test, y_test)
    
        tests = [
            (lambda x: x, "x.dat"),
            (lambda x: np.sin(np.pi * x), "sinx.dat"),
            (lambda x: np.sin(2 * np.pi * x), "sin2x.dat"),
            (lambda x: x * np.sin(2 * np.pi * x), "xsin2x.dat"),
        ]
        for u, fname in tests:
            if problem == "ode":
                test_u_ode(nn, system, T, m, model, data, u, fname)
    
    def main():
        # Problems:
        # - "lt": Legendre transform
        # - "ode": Antiderivative, Nonlinear ODE, Gravity pendulum
        # - "dr": Diffusion-reaction
        # - "cvc": Advection
        # - "advd": Advection-diffusion
        problem = "ode"
        T = 1
        system = ode_system(T)
    
        # Function space
        # space = FinitePowerSeries(N=100, M=1)
        # space = FiniteChebyshev(N=20, M=1)
        # space = GRF(2, length_scale=0.2, N=2000, interp="cubic")  # "lt"
        space = GRF(1, length_scale=0.2, N=1000, interp="cubic")
        # space = GRF(T, length_scale=0.2, N=1000 * T, interp="cubic")
    
        # Hyperparameters
        m = 100
        num_train = 10000
        num_test = 100000
        lr = 0.001
        epochs = 50000
    
        # Network
        nn = "opnn"
        activation = "relu"
        initializer = "Glorot normal"  # "He normal" or "Glorot normal"
        dim_x = 1 if problem in ["ode", "lt"] else 2
        if nn == "opnn":
            net = dde.maps.DeepONetCartesianProd(
                [m, 40, 40],
                [dim_x, 40, 40],
                activation,
                initializer,
                # use_bias=True,
                # stacked=False,
            )
        elif nn == "fnn":
            net = dde.maps.FNN([m + dim_x] + [100] * 2 + [1], activation, initializer)
        elif nn == "resnet":
            net = dde.maps.ResNet(m + dim_x, 1, 128, 2, activation, initializer)
    
        run(problem, system, space, T, m, nn, net, lr, epochs, num_train, num_test)
    
    if __name__ == "__main__":
        main()
    
    

    Taking a look briefly at the exports in deepxde.nn.tensorflow I think that the Installation Guide should point to DeepONetCartesianProd and not DeepONet, right? Or am I missing anything? Now even if I make that change, the code is not runnable as is. Do you have any inputs?

    I will keep exploring the library, but it'd be great if you could clarify a bit on how to make the above snippet runnable.

    Thanks!

    opened by bhaveshshrimali 5
  • The data shape of the triplet if the number of input functions of branch nets is more than one?

    The data shape of the triplet if the number of input functions of branch nets is more than one?

    Hello: First of all, thank you very much for your excellent work! I have met a little problem. How to reshape the dataset (or change the code) if the number of input functions of branch nets is more than one? Just like G_phi in this paper("DeepM&Mnet: Inferring the electroconvection multiphysics fields..."), (c+, c-) -->phi. Or like G_rhoNO in this paper("DeepM&Mnet for hypersonics..."), (rho_N2, rho_O2) --> rho_NO. Specifically, what about the data shape of the triplet? Thanks in advance.

    opened by WangYicunZJU 4
  • Could a function of Lp space(p=2) be branch input?

    Could a function of Lp space(p=2) be branch input?

    Dear Professor.Lulu:

    As title. I notice you employ KL expansion in Learning stochastic operators, and use N eigenfunctions scaled by the eigenvalues as the input of branch net.

    My problem is about Sturm-Liouville problem, the solution is also consist of eigenfunctions and eigenvalues.

    Is that mean i should train model for the problem with different number of eigenvalues, and the input of branch net is N orthogonal functions? If there is no random process, the input of the branch net is function u of variable x, the function is sample from Lp space(p=2) .It's OK use the function u(x) instead of Fourier-Bessel expansion?

    opened by xuliang5115 3
  • How to solve 3d ode equations

    How to solve 3d ode equations

    Hi, First of all, thank you very much for your excellent work.

    In case Gravity pendulum, how do I print the predicted value of the variable s2?

    I also try to solve a 3D ode equation set, like ds1/dt=g1(x),ds2/dt=g2(x),ds3/dt=g3(x).It works very well, but only outputs the s1 in test ode function.

    Looking forward to your reply.

    Thank you very much.

    opened by ajogajog 3
  • AttributeError: 'Triple' object has no attribute 'transform_inputs'

    AttributeError: 'Triple' object has no attribute 'transform_inputs'

    Any idea how to resolve the following error? This is for the default problem = "ode" test case. Backend: tensorflow.compat.v1 TensorFlow version: 2.6.2 Thanks.

    Restoring model from model/model.ckpt-37000.ckpt ...

    Traceback (most recent call last): File "deeponet_pde.py", line 285, in main() File "deeponet_pde.py", line 281, in main run(problem, system, space, T, m, nn, net, lr, epochs, num_train, num_test) File "deeponet_pde.py", line 177, in run safe_test(model, data, X_test, y_test) File "/[HOME-PATH]/tests/deeponet/src/utils.py", line 56, in safe_test y_pred.append(model.predict(data.transform_inputs(X_add))) AttributeError: 'Triple' object has no attribute 'transform_inputs'

    opened by cfd-ai 2
  • Number of parameters in `seq2seq` scales with the length**2 of the sequence

    Number of parameters in `seq2seq` scales with the length**2 of the sequence

    https://github.com/lululxvi/deeponet/blob/44e7b5a2356b0e6e00ad101e897342a2f1e87c0b/seq2seq/learner/nn/seq2seq.py#L58-L59

    This does not seem right, considering DeepONet does not have such kind of dependence.

    https://github.com/lululxvi/deeponet/blob/44e7b5a2356b0e6e00ad101e897342a2f1e87c0b/seq2seq/learner/nn/deeponet.py#L39-L41

    opened by scaomath 1
  • How to use the trained model to predict without training?

    How to use the trained model to predict without training?

    Hello Dr. Lu, Thank you very much for putting forward this DeepONet for our study.I'm very interested in DeepONet . But now I have a problem.I spent a lot of time generating datasets and training on reaction-diffusion equations. I save the training data set and the best model after training. But how can I use this optimal model to predict the solution of the reaction-diffusion equation without training? How can I modify the code in deeponet_pde.py? Thank you very much for your time.

    opened by blackyichao 1
  • Dimension issue in generating operator

    Dimension issue in generating operator "system.py".

    Hi, First of all, thank you very much for your excellent work.

    I tried to test the dimension-match in system.py for the case of simple linear ODE with the following parameters:

    • The space that I choose is Finite Power Series with N = 100 and M = 1
    • m: the number of sensors (So I get the sensors to be an array of m-by-1) Choose m = 50
    • m_u: the number of u's ( the sample u is an array of m_u-by-m) Choose m_u = 60
    • m_y: the number of y's ( an array of m_y-by-1) Choose m_y = 40

    However, I get the dimension for s which value of G(u)(y) is an array of 40-by-1. Should this value have to be an array of (m_u * m_y)-by-1 which is in this case 2400-by-1?

    This is my modified part:

    def gen_operator_data(self,space,m, m_u, m_y):
            print("GENERATING THE DATA...", flush = True)           
            features= space.random(m_u) # build the features (m_u, N) check
            sensors = np.linspace(0, self.T, m)[:, None] # build the sensors (m,1) check 
            u = space.eval_u(features_train, sensors) # get the sample u's (m_u, m) check
            x  = self.T * np.random.rand(m_y)[:,None] # get the sample y's (m_y, 1) check
            s = self.eval_op_space(space, features, x) # get the values s'  
            Xdata = [u, x] 
            Ydata = s
            return Xdata, Ydata
    

    I look forward to hearing from you soon.

    Best Regards.

    opened by phuongle0701 1
  • input data format for OpNN

    input data format for OpNN

    Hi Lu, Thanks for your work. I am trying to make some data for OpNN training. just for clarifying, should the input data be [initial condition, position] and the label be [true value w.r.t. input]?

    opened by ChironJC 1
  • two-dimensional PDE problem of deeponet

    two-dimensional PDE problem of deeponet

    Thank you for your previous answer. The problem of model recovery has been solved. But I have another question.Can we use deeponet to solve the two-dimensional PDE problem? If so, can I input a list of X and Y in the Trunk net?

    opened by blackyichao 0
  • If the input of the trunk net could be set to be different

    If the input of the trunk net could be set to be different

    Hello Dr. Lu,

    I am Bo Chen from SJTU. We had some communication before in online conference, and I have read your code of the article"A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data". It is really wonderful work and I have learnt a lot from it.

    But it can be found that in the cases you offered, the input of the trunk net is always the same distribution for every input of the branch net so that the data can be set by "dde.data.TripleCartesianProd()" with a grid of only one distribution.

    However in my application, the input of the trunk net must be different distribution because of the data limitation of labels. This situation is just
    mentioned in your article as case I or case II ,which means different u have the different locations of ξ.

    image

    I have made same attempts but it seems that the shape of data_trunknet is fixed and hard to be changed. And I also see a type of error "ValueError("DeepONet does not support setting trunk net input.")" in the code even though I did not meet this kind of error in my attempts.

    So I wonder if there is a method of that could feed the input of the trunk net with different distribution to DEEPONET. If possible, could you give me some guidance or a simple demo is better.

    Thanks very much!

    Looking forword to your reply!

    opened by cb-sjtu 7
  • High test loss for nonlinear ODE case

    High test loss for nonlinear ODE case

    I modified both main() and ode_system() as stated in the instructions, but I get a very high test loss. Do I have to modify other functions? I also increased the number of epochs to 10^5 as pointed in the supplementary information of the cited paper (a very interesting work I have to say). Thank you in advance.

    Additional information: Modified lines (all in src/deeponet_pde.py): line 104: uncommented after having commented line 102 line 231: be sure to have problem="ode" line 260: increased epochs to 100000

    Example of output:

    Step      Train loss    Test loss     Test metric   
    0         [3.68e-01]    [1.74e+25]    [3.55e-01]    
    1000      [3.08e-03]    [1.74e+25]    [2.05e-03]    
    2000      [8.55e-04]    [1.74e+25]    [8.00e-04]    
    3000      [3.57e-04]    [1.74e+25]    [5.24e-04]    
    4000      [2.06e-04]    [1.74e+25]    [3.58e-04]    
    5000      [1.65e-04]    [1.74e+25]    [3.22e-04]    
    6000      [1.43e-04]    [1.74e+25]    [2.83e-04]    
    7000      [1.28e-04]    [1.74e+25]    [2.47e-04]    
    8000      [1.01e-04]    [1.74e+25]    [2.26e-04]    
    9000      [8.53e-05]    [1.74e+25]    [2.00e-04]    
    10000     [7.95e-05]    [1.74e+25]    [1.87e-04]    
    11000     [7.90e-05]    [1.74e+25]    [1.80e-04]    
    12000     [9.09e-05]    [1.74e+25]    [1.88e-04]    
    13000     [9.10e-05]    [1.74e+25]    [1.69e-04]    
    14000     [5.61e-05]    [1.74e+25]    [1.45e-04]    
    15000     [5.24e-05]    [1.74e+25]    [1.40e-04]    
    16000     [5.50e-05]    [1.74e+25]    [1.44e-04]    
    17000     [5.00e-05]    [1.74e+25]    [1.33e-04]    
    18000     [5.21e-05]    [1.74e+25]    [1.39e-04]    
    19000     [4.54e-05]    [1.74e+25]    [1.31e-04]    
    20000     [1.13e-04]    [1.74e+25]    [1.78e-04]    
    21000     [4.39e-05]    [1.74e+25]    [1.31e-04]    
    22000     [5.68e-05]    [1.74e+25]    [1.45e-04]    
    23000     [4.37e-05]    [1.74e+25]    [1.31e-04]    
    24000     [4.18e-05]    [1.74e+25]    [1.28e-04]    
    25000     [4.00e-05]    [1.74e+25]    [1.28e-04]    
    26000     [6.98e-05]    [1.74e+25]    [1.55e-04]    
    27000     [7.45e-05]    [1.74e+25]    [1.58e-04]    
    28000     [4.10e-05]    [1.74e+25]    [1.22e-04]    
    29000     [3.62e-05]    [1.74e+25]    [1.20e-04]    
    30000     [5.16e-05]    [1.74e+25]    [1.36e-04]    
    31000     [3.60e-05]    [1.74e+25]    [1.19e-04]    
    32000     [3.82e-05]    [1.74e+25]    [1.20e-04]    
    33000     [3.33e-05]    [1.74e+25]    [1.14e-04]    
    34000     [3.36e-05]    [1.74e+25]    [1.14e-04]    
    35000     [3.24e-05]    [1.74e+25]    [1.11e-04]    
    36000     [3.15e-05]    [1.74e+25]    [1.12e-04]    
    37000     [3.14e-05]    [1.74e+25]    [1.08e-04]    
    38000     [3.78e-05]    [1.74e+25]    [1.18e-04]    
    39000     [3.40e-05]    [1.74e+25]    [1.09e-04]    
    40000     [4.34e-05]    [1.74e+25]    [1.14e-04]    
    41000     [3.58e-05]    [1.74e+25]    [1.09e-04]    
    42000     [2.89e-05]    [1.74e+25]    [1.04e-04]    
    43000     [2.99e-05]    [1.74e+25]    [1.09e-04]    
    44000     [4.13e-05]    [1.74e+25]    [1.10e-04]    
    45000     [2.72e-05]    [1.74e+25]    [1.03e-04]    
    46000     [2.71e-05]    [1.74e+25]    [1.03e-04]    
    47000     [2.88e-05]    [1.74e+25]    [1.02e-04]    
    48000     [5.22e-05]    [1.74e+25]    [1.33e-04]    
    49000     [2.55e-05]    [1.74e+25]    [9.98e-05]    
    50000     [2.92e-05]    [1.74e+25]    [1.01e-04]    
    51000     [2.63e-05]    [1.74e+25]    [9.88e-05]    
    52000     [2.39e-05]    [1.74e+25]    [9.90e-05]    
    53000     [2.76e-05]    [1.74e+25]    [1.02e-04]    
    54000     [3.15e-05]    [1.74e+25]    [9.99e-05]    
    55000     [3.11e-05]    [1.74e+25]    [1.11e-04]    
    56000     [2.47e-05]    [1.74e+25]    [9.72e-05]    
    57000     [3.14e-05]    [1.74e+25]    [1.08e-04]    
    58000     [3.34e-05]    [1.74e+25]    [1.01e-04]    
    59000     [2.54e-05]    [1.74e+25]    [9.68e-05]    
    60000     [3.11e-05]    [1.74e+25]    [9.88e-05]    
    61000     [4.05e-05]    [1.74e+25]    [1.20e-04]    
    62000     [4.36e-05]    [1.74e+25]    [1.22e-04]    
    63000     [2.21e-05]    [1.74e+25]    [9.88e-05]    
    64000     [2.31e-05]    [1.74e+25]    [9.72e-05]    
    65000     [3.04e-05]    [1.74e+25]    [9.75e-05]    
    66000     [2.57e-05]    [1.74e+25]    [9.81e-05]    
    67000     [2.77e-05]    [1.74e+25]    [9.60e-05]    
    68000     [2.22e-05]    [1.74e+25]    [9.49e-05]    
    69000     [2.21e-05]    [1.74e+25]    [9.86e-05]    
    70000     [2.57e-05]    [1.74e+25]    [9.89e-05]    
    71000     [2.21e-05]    [1.74e+25]    [9.48e-05]    
    72000     [2.11e-05]    [1.74e+25]    [9.38e-05]    
    73000     [2.57e-05]    [1.74e+25]    [1.02e-04]    
    74000     [3.00e-05]    [1.74e+25]    [1.09e-04]    
    75000     [2.15e-05]    [1.74e+25]    [9.34e-05]    
    76000     [2.06e-05]    [1.74e+25]    [9.62e-05]    
    77000     [3.68e-05]    [1.74e+25]    [1.16e-04]    
    78000     [1.95e-05]    [1.74e+25]    [9.37e-05]    
    79000     [2.38e-05]    [1.74e+25]    [9.98e-05]    
    80000     [6.12e-05]    [1.74e+25]    [1.42e-04]    
    81000     [2.97e-05]    [1.74e+25]    [1.08e-04]    
    82000     [2.12e-05]    [1.74e+25]    [9.13e-05]    
    83000     [2.11e-05]    [1.74e+25]    [9.13e-05]    
    84000     [1.92e-05]    [1.74e+25]    [9.08e-05]    
    85000     [4.75e-05]    [1.74e+25]    [1.28e-04]    
    86000     [6.35e-05]    [1.74e+25]    [1.44e-04]    
    87000     [1.88e-05]    [1.74e+25]    [9.50e-05]    
    88000     [1.85e-05]    [1.74e+25]    [9.09e-05]    
    89000     [1.74e-05]    [1.74e+25]    [9.12e-05]    
    90000     [1.75e-05]    [1.74e+25]    [9.28e-05]    
    91000     [3.79e-05]    [1.74e+25]    [1.05e-04]    
    92000     [1.98e-05]    [1.74e+25]    [9.06e-05]    
    93000     [2.04e-05]    [1.74e+25]    [9.09e-05]    
    94000     [2.14e-05]    [1.74e+25]    [9.92e-05]    
    95000     [1.70e-05]    [1.74e+25]    [9.22e-05]    
    96000     [3.40e-05]    [1.74e+25]    [9.47e-05]    
    97000     [1.80e-05]    [1.74e+25]    [9.49e-05]    
    98000     [1.65e-05]    [1.74e+25]    [9.20e-05]    
    99000     [1.67e-05]    [1.74e+25]    [8.97e-05]    
    100000    [1.65e-05]    [1.74e+25]    [9.20e-05]    
    
    Best model at step 98000:
      train loss: 1.65e-05
      test loss: 1.74e+25
      test metric: [9.20e-05]
    
    
    opened by edoardo100 1
  • AttributeError: 'Triple' object has no attribute 'transform_inputs'

    AttributeError: 'Triple' object has no attribute 'transform_inputs'

    model.restore("model/model.ckpt-" + str(train_state.best_step) + ".ckpt", verbose=1) #model.restore("model/model.ckpt-" + str(train_state.best_step), verbose=1) safe_test(model, data, X_test, y_test)

    Saving loss history to D:\LULU\deeponet-master1\deeponet-master\src\loss.dat ... Error: The network has multiple inputs, and saving such result han't been implemented. Error: The network has multiple inputs, and plotting such result han't been implemented. Restoring model from model/model.ckpt-5000.ckpt ...

    Traceback (most recent call last): File "D:/LULU/deeponet-master1/deeponet-master/src/deeponet_pde.py", line 285, in main() File "D:\LULU/deeponet-master1/deeponet-master/src/deeponet_pde.py", line 281, in main run(problem, system, space, T, m, nn, net, lr, epochs, num_train, num_test) File "D:\LULU/deeponet-master1/deeponet-master/src/deeponet_pde.py", line 177, in run safe_test(model, data, X_test, y_test) File "D:\LULU\deeponet-master1\deeponet-master\src\utils.py", line 56, in safe_test y_pred.append(model.predict(data.transform_inputs(X_add))) AttributeError: 'Triple' object has no attribute 'transform_inputs'

    进程已结束,退出代码1

    opened by jinjunyonggzh 1
  • How DeepONet provides multiple outputs

    How DeepONet provides multiple outputs

    Professor Lu: For a complex output problem, I divided into two networks with the same structure and hyperparameters for training. The previous issue has been resolved. So, i need put model1 and model2 in two files. I use step method to reduce the dimension of Trunk net, the input of each step is the output of the two networks of the previous step. So there is a bug. I think it is necessary to train a network with two outputs at the same time. May i ask some relevant examples so i can change my code?

    opened by xuliang5115 2
  • Questions about the difference PINN and DeepXDE

    Questions about the difference PINN and DeepXDE

    Dear Prof. Lu, Thanks for your time for looking at this question.

    1. Could you tell me is the DeepXDE is one type of PINN? But I didn't find any code about using control equation to describe the loss in DeepXDE.
    2. Currently, I am solving a equations contained possion and transport equations. I want to obtain the dynamic process of the electron density change. Should I use the (x,y,t) and (E,ne,np,nn) as the input and output, respectively. Can the DeepXDE predict the extra time results without trained time data as input.
    opened by pengcz 1
  • UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 38: invalid continuation byte

    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 38: invalid continuation byte

    Hello, professor. I have some porblem.What's the way you coded? I have the following error while running the code.

    Traceback (most recent call last): File "E:/DeepONet/deeponet-master/src/deeponet_pde.py", line 285, in main() File "E:/DeepONet/deeponet-master/src/deeponet_pde.py", line 281, in main run(problem, system, space, T, m, nn, net, lr, epochs, num_train, num_test) File "E:/DeepONet/deeponet-master/src/deeponet_pde.py", line 176, in run model.restore("model/model.ckpt-" + str(train_state.best_step), verbose=1) File "D:\ProgramData\Anaconda3\envs\donlu\lib\site-packages\deepxde\model.py", line 377, in restore self.saver.restore(self.sess, save_path) File "D:\ProgramData\Anaconda3\envs\donlu\lib\site-packages\tensorflow\python\training\saver.py", line 1289, in restore if not checkpoint_management.checkpoint_exists_internal(checkpoint_prefix): File "D:\ProgramData\Anaconda3\envs\donlu\lib\site-packages\tensorflow\python\training\checkpoint_management.py", line 383, in checkpoint_exists_internal if file_io.get_matching_files(pathname): File "D:\ProgramData\Anaconda3\envs\donlu\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 350, in get_matching_files return get_matching_files_v2(filename) File "D:\ProgramData\Anaconda3\envs\donlu\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 371, in get_matching_files_v2 compat.as_bytes(pattern)) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 38: invalid continuation byte

    Process finished with exit code 1

    opened by DaJiang7 2
Releases(v1.0.0)
Owner
Lu Lu
Applied Mathematics Instructor, Massachusetts Institute of Technology
Lu Lu
pytorch implementation for Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network arXiv:1609.04802

PyTorch SRResNet Implementation of Paper: "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network"(https://arxiv.org/abs

Jiu XU 436 Jan 09, 2023
Optimal space decomposition based-product quantization for approximate nearest neighbor search

Optimal space decomposition based-product quantization for approximate nearest neighbor search Abstract Product quantization(PQ) is an effective neare

Mylove 1 Nov 19, 2021
Neighborhood Reconstructing Autoencoders

Neighborhood Reconstructing Autoencoders The official repository for Neighborhood Reconstructing Autoencoders (Lee, Kwon, and Park, NeurIPS 2021). T

Yonghyeon Lee 24 Dec 14, 2022
Python Jupyter kernel using Poetry for reproducible notebooks

Poetry Kernel Use per-directory Poetry environments to run Jupyter kernels. No need to install a Jupyter kernel per Python virtual environment! The id

Pathbird 204 Jan 04, 2023
a curated list of docker-compose files prepared for testing data engineering tools, databases and open source libraries.

data-services A repository for storing various Data Engineering docker-compose files in one place. How to use it ? Set the required settings in .env f

BigData.IR 525 Dec 03, 2022
Code for “ACE-HGNN: Adaptive Curvature ExplorationHyperbolic Graph Neural Network”

ACE-HGNN: Adaptive Curvature Exploration Hyperbolic Graph Neural Network This repository is the implementation of ACE-HGNN in PyTorch. Environment pyt

9 Nov 28, 2022
Code Release for ICCV 2021 (oral), "AdaFit: Rethinking Learning-based Normal Estimation on Point Clouds"

AdaFit: Rethinking Learning-based Normal Estimation on Point Clouds (ICCV 2021 oral) **Project Page | Arxiv ** Runsong Zhu¹, Yuan Liu², Zhen Dong¹, Te

40 Dec 30, 2022
Serve TensorFlow ML models with TF-Serving and then create a Streamlit UI to use them

TensorFlow Serving + Streamlit! ✨ 🖼️ Serve TensorFlow ML models with TF-Serving and then create a Streamlit UI to use them! This is a pretty simple S

Álvaro Bartolomé 18 Jan 07, 2023
Densely Connected Convolutional Networks, In CVPR 2017 (Best Paper Award).

Densely Connected Convolutional Networks (DenseNets) This repository contains the code for DenseNet introduced in the following paper Densely Connecte

Zhuang Liu 4.5k Jan 03, 2023
Res2Net for Instance segmentation and Object detection using MaskRCNN

Res2Net for Instance segmentation and Object detection using MaskRCNN Since the MaskRCNN-benchmark of facebook is deprecated, we suggest to use our mm

Res2Net Applications 55 Oct 30, 2022
2.86% and 15.85% on CIFAR-10 and CIFAR-100

Shake-Shake regularization This repository contains the code for the paper Shake-Shake regularization. This arxiv paper is an extension of Shake-Shake

Xavier Gastaldi 294 Nov 22, 2022
List of content farm sites like g.penzai.com.

内容农场网站清单 Google 中文搜索结果包含了相当一部分的内容农场式条目,比如「小 X 知识网」「小 X 百科网」。此种链接常会 302 重定向其主站,页面内容为自动生成,大量堆叠关键字,揉杂一些爬取到的内容,完全不具可读性和参考价值。 尤为过分的是,该类网站可能有成千上万个分身域名被 Goog

WDMPA 541 Jan 03, 2023
Contrastive Fact Verification

VitaminC This repository contains the dataset and models for the NAACL 2021 paper: Get Your Vitamin C! Robust Fact Verification with Contrastive Evide

47 Dec 19, 2022
A modification of Daniel Russell's notebook merged with Katherine Crowson's hq-skip-net changes

Edits made to this repo by Katherine Crowson I have added several features to this repository for use in creating higher quality generative art (featu

Paul Fishwick 10 May 07, 2022
Official implementation of the MM'21 paper Constrained Graphic Layout Generation via Latent Optimization

[MM'21] Constrained Graphic Layout Generation via Latent Optimization This repository provides the official code for the paper "Constrained Graphic La

Kotaro Kikuchi 73 Dec 27, 2022
IEGAN — Official PyTorch Implementation Independent Encoder for Deep Hierarchical Unsupervised Image-to-Image Translation

IEGAN — Official PyTorch Implementation Independent Encoder for Deep Hierarchical Unsupervised Image-to-Image Translation Independent Encoder for Deep

30 Nov 05, 2022
WaveFake: A Data Set to Facilitate Audio DeepFake Detection

WaveFake: A Data Set to Facilitate Audio DeepFake Detection This is the code repository for our NeurIPS 2021 (Track on Datasets and Benchmarks) paper

Chair for Sys­tems Se­cu­ri­ty 27 Dec 22, 2022
🏃‍♀️ A curated list about human motion capture, analysis and synthesis.

Awesome Human Motion 🏃‍♀️ A curated list about human motion capture, analysis and synthesis. Contents Introduction Human Models Datasets Data Process

Dennis Wittchen 274 Dec 14, 2022
The PyTorch implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision.

DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision The PyTorch implementation of DiscoBox: Weakly Supe

Shiyi Lan 1 Oct 23, 2021
multimodal transformer

This repo holds the code to perform experiments with the multimodal autoregressive probabilistic model Transflower. Overview of the repo It is structu

Guillermo Valle 68 Dec 13, 2022