Quantcast

file sink

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

file sink

Mike F Nelson
Hello,

I am using Simphony and I have a question about file sinks for batch runs.  Saving results as text files seems straightforward, but I really want to have model output in netCDF.  I have written some netCDF outputters, which save my run data the way I want, but I’m running into some problems with where the results are saved in batch runs.  Currently the simulations files are saved in very unintuitive locations:

/private/var/folders/6h/psgxft793wg33tlfjyv367r4t2v5zn/T/simphony_model_(some horrible string of numbers)/instance_*/*.nc

when I do batch runs on my local machine.  I’m not sure how I will ever find these results on a remote machine when I try to do batch runs there.  I’m not looking forward to wasting a bunch of time searching for output files when I switch to a remote machine.  It would really like to be able to specify an output location.  It seems like this is possible using Repast’s file sinks, but I don’t want to output text files as they would be way to large.

Is there a way to specify where the model instances’ output files are saved?  Preferable using relative paths, but absolute paths would be ok too.

Thanks in advance.
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Repast-interest mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/repast-interest
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: file sink

srcnick
The Repast batch run GUI allows you to specify “optional output file patterns”. Files that match this pattern will be copied back to your local output directory in the same that file sink produced files are. So, if you specify a pattern for your netCDF files, you shouldn’t have to hunt for them in the temporary batch run directories. The “optional output file patterns” are described on page 4 of the batch run getting started guide. 


Hope this helps,

Nick


On Jan 11, 2017, at 11:37 AM, Mike F Nelson <[hidden email]> wrote:

Hello,

I am using Simphony and I have a question about file sinks for batch runs.  Saving results as text files seems straightforward, but I really want to have model output in netCDF.  I have written some netCDF outputters, which save my run data the way I want, but I’m running into some problems with where the results are saved in batch runs.  Currently the simulations files are saved in very unintuitive locations:

/private/var/folders/6h/psgxft793wg33tlfjyv367r4t2v5zn/T/simphony_model_(some horrible string of numbers)/instance_*/*.nc

when I do batch runs on my local machine.  I’m not sure how I will ever find these results on a remote machine when I try to do batch runs there.  I’m not looking forward to wasting a bunch of time searching for output files when I switch to a remote machine.  It would really like to be able to specify an output location.  It seems like this is possible using Repast’s file sinks, but I don’t want to output text files as they would be way to large.

Is there a way to specify where the model instances’ output files are saved?  Preferable using relative paths, but absolute paths would be ok too.

Thanks in advance.
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Repast-interest mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/repast-interest


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Repast-interest mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/repast-interest
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: file sink

Mike F Nelson
I am having a problem with getting batch runs to work.  My model launches and runs in the GUI, saving results in the appropriate files.  There are no warnings or errors when I run the model in the GUI.  When I try to run the batch runs launcher, i get these errors:

java.lang.ClassCastException: javax.swing.JCheckBox cannot be cast to javax.swing.JTextField
        at repast.simphony.batch.gui.ParameterInputPanel$ConstantInputPanel.setDefaultValue(ParameterInputPanel.java:374)
        at repast.simphony.batch.gui.ParameterInputPanel.update(ParameterInputPanel.java:108)
        at repast.simphony.batch.gui.ParameterInputPanel.<init>(ParameterInputPanel.java:79)
        at repast.simphony.batch.gui.BatchParamPanel.loadParameters(BatchParamPanel.java:281)
        at repast.simphony.batch.gui.BatchParamPanel.updateInputPanelsFromP(BatchParamPanel.java:163)
        at repast.simphony.batch.gui.BatchParamPanel.access$5(BatchParamPanel.java:157)
        at repast.simphony.batch.gui.BatchParamPanel$5.insertUpdate(BatchParamPanel.java:147)
        at javax.swing.text.AbstractDocument.fireInsertUpdate(AbstractDocument.java:201)
        at javax.swing.text.AbstractDocument.handleInsertString(AbstractDocument.java:748)
        at javax.swing.text.AbstractDocument.insertString(AbstractDocument.java:707)
        at javax.swing.text.PlainDocument.insertString(PlainDocument.java:130)
        at javax.swing.text.AbstractDocument.replace(AbstractDocument.java:669)
        at javax.swing.text.JTextComponent.setText(JTextComponent.java:1669)
        at repast.simphony.batch.gui.BatchParamPanel.update(BatchParamPanel.java:202)
        at repast.simphony.batch.gui.BatchConfigMediator.updateFromModel(BatchConfigMediator.java:284)
        at repast.simphony.batch.gui.BatchConfigMediator.createTabs(BatchConfigMediator.java:145)
        at repast.simphony.batch.gui.MainPanel.init(MainPanel.java:33)
        at repast.simphony.batch.gui.MainPanel.<init>(MainPanel.java:28)
        at repast.simphony.batch.gui.Main.run(Main.java:65)
        at repast.simphony.batch.gui.Main$1.run(Main.java:93)
        at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
        at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
        at java.awt.EventQueue.access$500(EventQueue.java:97)
        at java.awt.EventQueue$3.run(EventQueue.java:709)
        at java.awt.EventQueue$3.run(EventQueue.java:703)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
        at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)



The batch stand alone window opens and I can generate a batch parameters file.  When I try to stage the runs, it appears to stage them successfully, but then I get the following cascade of errors:




INFO  [AWT-EventQueue-0] 17:12:00,730 repast.simphony.batch.gui.BatchConfigMediator - Writing batch run config file to: /Users/mfnelson/github/MPB_RS/output/config.props
INFO  [AWT-EventQueue-0] 17:12:00,785 repast.simphony.batch.gui.BatchConfigMediator - Unrolling batch parameter file:
        /Users/mfnelson/github/MPB_RS/batch/batch_params.xml to
        /var/folders/6h/psgxft793wg33tlfjyv367r4t2v5zn/T/unrolledParamFile.txt
INFO  [SwingWorker-pool-1-thread-1] 17:12:13,067 repast.simphony.batch.ssh.LocalSession -
        Copying locally /var/folders/6h/psgxft793wg33tlfjyv367r4t2v5zn/T/mfnelson_localhost6837636286075138521.zip to
/var/folders/6h/psgxft793wg33tlfjyv367r4t2v5zn/T/simphony_model_1484701928156 ...
INFO  [SwingWorker-pool-1-thread-1] 17:12:13,665 repast.simphony.batch.ssh.LocalSession - Copying Finished.
INFO  [SwingWorker-pool-1-thread-1] 17:12:13,667 repast.simphony.batch.ssh.SessionsDriver - Initialization Time: 0.0918
INFO  [SwingWorker-pool-1-thread-1] 17:12:13,667 repast.simphony.batch.ssh.LocalSession - Unzipping model /var/folders/6h/psgxft793wg33tlfjyv367r4t2v5zn/T/simphony_model_1484701928156/mfnelson_localhost6837636286075138521.zip
INFO  [SwingWorker-pool-1-thread-1] 17:12:14,409 repast.simphony.batch.ssh.LocalSession - Running model on localhost ...
INFO  [Thread-1] 17:12:36,211 repast.simphony.batch.ssh.LocalSession - ERROR [main] 17:12:36,208 repast.simphony.batch.InstanceRunner - Error while running model
INFO  [Thread-1] 17:12:36,211 repast.simphony.batch.ssh.LocalSession - java.lang.ArrayIndexOutOfBoundsException: 1
INFO  [Thread-1] 17:12:36,211 repast.simphony.batch.ssh.LocalSession - at repast.simphony.batch.parameter.ParameterLineParser.parse(ParameterLineParser.java:55)
INFO  [Thread-1] 17:12:36,212 repast.simphony.batch.ssh.LocalSession - at repast.simphony.batch.InstanceRunner.run(InstanceRunner.java:193)
INFO  [Thread-1] 17:12:36,212 repast.simphony.batch.ssh.LocalSession - at repast.simphony.batch.InstanceRunner.main(InstanceRunner.java:217)
INFO  [Thread-1] 17:12:36,212 repast.simphony.batch.ssh.LocalSession - Error while running model
INFO  [Thread-1] 17:12:36,212 repast.simphony.batch.ssh.LocalSession - java.lang.ArrayIndexOutOfBoundsException: 1
INFO  [Thread-1] 17:12:36,212 repast.simphony.batch.ssh.LocalSession - at repast.simphony.batch.parameter.ParameterLineParser.parse(ParameterLineParser.java:55)
INFO  [Thread-1] 17:12:36,212 repast.simphony.batch.ssh.LocalSession - at repast.simphony.batch.InstanceRunner.run(InstanceRunner.java:193)
INFO  [Thread-1] 17:12:36,212 repast.simphony.batch.ssh.LocalSession - at repast.simphony.batch.InstanceRunner.main(InstanceRunner.java:217)
INFO  [pool-2-thread-1] 17:13:14,421 repast.simphony.batch.ssh.LocalDonePoller - Polled /var/folders/6h/psgxft793wg33tlfjyv367r4t2v5zn/T/simphony_model_1484701928156 for DONE:  yes
INFO  [SwingWorker-pool-1-thread-1] 17:13:14,422 repast.simphony.batch.ssh.SessionsDriver - Run Time: 1.0126
INFO  [SwingWorker-pool-1-thread-1] 17:13:14,427 repast.simphony.batch.ssh.LocalSession - Finding output on localhost in /var/folders/6h/psgxft793wg33tlfjyv367r4t2v5zn/T/simphony_model_1484701928156
INFO  [SwingWorker-pool-1-thread-1] 17:13:14,450 repast.simphony.batch.ssh.SessionsDriver - Aggregating output into /Users/mfnelson/github/MPB_RS/output
INFO  [SwingWorker-pool-1-thread-1] 17:13:14,450 repast.simphony.batch.ssh.SessionsDriver - Get Output Time: 0.0005
INFO  [SwingWorker-pool-1-thread-1] 17:13:14,450 repast.simphony.batch.ssh.SessionsDriver - Finished. Elapsed Time: 1.1058


I have already tried changing the boolean parameters to String within parameters.xml and then parsing them as booleans within the model code.  That eliminated the first set of errors, but I still got the second sting of errors and the runs failed again.

The batch runs previously worked in Repast 2.3.1, and I’m now using 2.4.0.  I had made a number of edits in between the time I updated to 2.4.0 and when I tried do batch runs.  I have not, however, made any changes since that time to parameters.xml or the code that reads in the parameters.

Any help would be appreciated.  I’m hoping not to have to go back to 2.3.1 because I’m worried that will entail a lot of work (and time).

Thank you,

Mike





------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Repast-interest mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/repast-interest
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: file sink

srcnick
Mike,

From the stack trace for the second issue it looks like there’s some problem with the format of your unrolledParamFile.txt. The elements in that need to be tab separated. The line in the stack trace splits a string by a tab into two elements. If there is no tab character then you’ll get the array out of bounds exception as the array that is the result of the split has only a single element. Is it possible you’ve opened unrolledParamFile.txt in some editor that automatically replaces tabs with spaces?

The first issue is a known bug that’s been fixed in the development version. I can send you some updated .class files that contain the fix you want. The workaround as you’ve discovered is to replace the boolean parameters with string ones.

Nick

>
>
> I am having a problem with getting batch runs to work.  My model launches and runs in the GUI, saving results in the appropriate files.  There are no warnings or errors when I run the model in the GUI.  When I try to run the batch runs launcher, i get these errors:
>
> java.lang.ClassCastException: javax.swing.JCheckBox cannot be cast to javax.swing.JTextField
> at repast.simphony.batch.gui.ParameterInputPanel$ConstantInputPanel.setDefaultValue(ParameterInputPanel.java:374)
> at repast.simphony.batch.gui.ParameterInputPanel.update(ParameterInputPanel.java:108)
> at repast.simphony.batch.gui.ParameterInputPanel.<init>(ParameterInputPanel.java:79)
> at repast.simphony.batch.gui.BatchParamPanel.loadParameters(BatchParamPanel.java:281)
> at repast.simphony.batch.gui.BatchParamPanel.updateInputPanelsFromP(BatchParamPanel.java:163)
> at repast.simphony.batch.gui.BatchParamPanel.access$5(BatchParamPanel.java:157)
> at repast.simphony.batch.gui.BatchParamPanel$5.insertUpdate(BatchParamPanel.java:147)
> at javax.swing.text.AbstractDocument.fireInsertUpdate(AbstractDocument.java:201)
> at javax.swing.text.AbstractDocument.handleInsertString(AbstractDocument.java:748)
> at javax.swing.text.AbstractDocument.insertString(AbstractDocument.java:707)
> at javax.swing.text.PlainDocument.insertString(PlainDocument.java:130)
> at javax.swing.text.AbstractDocument.replace(AbstractDocument.java:669)
> at javax.swing.text.JTextComponent.setText(JTextComponent.java:1669)
> at repast.simphony.batch.gui.BatchParamPanel.update(BatchParamPanel.java:202)
> at repast.simphony.batch.gui.BatchConfigMediator.updateFromModel(BatchConfigMediator.java:284)
> at repast.simphony.batch.gui.BatchConfigMediator.createTabs(BatchConfigMediator.java:145)
> at repast.simphony.batch.gui.MainPanel.init(MainPanel.java:33)
> at repast.simphony.batch.gui.MainPanel.<init>(MainPanel.java:28)
> at repast.simphony.batch.gui.Main.run(Main.java:65)
> at repast.simphony.batch.gui.Main$1.run(Main.java:93)
> at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
> at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
> at java.awt.EventQueue.access$500(EventQueue.java:97)
> at java.awt.EventQueue$3.run(EventQueue.java:709)
> at java.awt.EventQueue$3.run(EventQueue.java:703)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
> at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
> at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
> at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
> at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
> at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
> at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
> at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
>
>
>
> The batch stand alone window opens and I can generate a batch parameters file.  When I try to stage the runs, it appears to stage them successfully, but then I get the following cascade of errors:
>
>
>
>
> INFO  [AWT-EventQueue-0] 17:12:00,730 repast.simphony.batch.gui.BatchConfigMediator - Writing batch run config file to: /Users/mfnelson/github/MPB_RS/output/config.props
> INFO  [AWT-EventQueue-0] 17:12:00,785 repast.simphony.batch.gui.BatchConfigMediator - Unrolling batch parameter file:
> /Users/mfnelson/github/MPB_RS/batch/batch_params.xml to
> /var/folders/6h/psgxft793wg33tlfjyv367r4t2v5zn/T/unrolledParamFile.txt
> INFO  [SwingWorker-pool-1-thread-1] 17:12:13,067 repast.simphony.batch.ssh.LocalSession -
> Copying locally /var/folders/6h/psgxft793wg33tlfjyv367r4t2v5zn/T/mfnelson_localhost6837636286075138521.zip to
> /var/folders/6h/psgxft793wg33tlfjyv367r4t2v5zn/T/simphony_model_1484701928156 ...
> INFO  [SwingWorker-pool-1-thread-1] 17:12:13,665 repast.simphony.batch.ssh.LocalSession - Copying Finished.
> INFO  [SwingWorker-pool-1-thread-1] 17:12:13,667 repast.simphony.batch.ssh.SessionsDriver - Initialization Time: 0.0918
> INFO  [SwingWorker-pool-1-thread-1] 17:12:13,667 repast.simphony.batch.ssh.LocalSession - Unzipping model /var/folders/6h/psgxft793wg33tlfjyv367r4t2v5zn/T/simphony_model_1484701928156/mfnelson_localhost6837636286075138521.zip
> INFO  [SwingWorker-pool-1-thread-1] 17:12:14,409 repast.simphony.batch.ssh.LocalSession - Running model on localhost ...
> INFO  [Thread-1] 17:12:36,211 repast.simphony.batch.ssh.LocalSession - ERROR [main] 17:12:36,208 repast.simphony.batch.InstanceRunner - Error while running model
> INFO  [Thread-1] 17:12:36,211 repast.simphony.batch.ssh.LocalSession - java.lang.ArrayIndexOutOfBoundsException: 1
> INFO  [Thread-1] 17:12:36,211 repast.simphony.batch.ssh.LocalSession - at repast.simphony.batch.parameter.ParameterLineParser.parse(ParameterLineParser.java:55)
> INFO  [Thread-1] 17:12:36,212 repast.simphony.batch.ssh.LocalSession - at repast.simphony.batch.InstanceRunner.run(InstanceRunner.java:193)
> INFO  [Thread-1] 17:12:36,212 repast.simphony.batch.ssh.LocalSession - at repast.simphony.batch.InstanceRunner.main(InstanceRunner.java:217)
> INFO  [Thread-1] 17:12:36,212 repast.simphony.batch.ssh.LocalSession - Error while running model
> INFO  [Thread-1] 17:12:36,212 repast.simphony.batch.ssh.LocalSession - java.lang.ArrayIndexOutOfBoundsException: 1
> INFO  [Thread-1] 17:12:36,212 repast.simphony.batch.ssh.LocalSession - at repast.simphony.batch.parameter.ParameterLineParser.parse(ParameterLineParser.java:55)
> INFO  [Thread-1] 17:12:36,212 repast.simphony.batch.ssh.LocalSession - at repast.simphony.batch.InstanceRunner.run(InstanceRunner.java:193)
> INFO  [Thread-1] 17:12:36,212 repast.simphony.batch.ssh.LocalSession - at repast.simphony.batch.InstanceRunner.main(InstanceRunner.java:217)
> INFO  [pool-2-thread-1] 17:13:14,421 repast.simphony.batch.ssh.LocalDonePoller - Polled /var/folders/6h/psgxft793wg33tlfjyv367r4t2v5zn/T/simphony_model_1484701928156 for DONE:  yes
> INFO  [SwingWorker-pool-1-thread-1] 17:13:14,422 repast.simphony.batch.ssh.SessionsDriver - Run Time: 1.0126
> INFO  [SwingWorker-pool-1-thread-1] 17:13:14,427 repast.simphony.batch.ssh.LocalSession - Finding output on localhost in /var/folders/6h/psgxft793wg33tlfjyv367r4t2v5zn/T/simphony_model_1484701928156
> INFO  [SwingWorker-pool-1-thread-1] 17:13:14,450 repast.simphony.batch.ssh.SessionsDriver - Aggregating output into /Users/mfnelson/github/MPB_RS/output
> INFO  [SwingWorker-pool-1-thread-1] 17:13:14,450 repast.simphony.batch.ssh.SessionsDriver - Get Output Time: 0.0005
> INFO  [SwingWorker-pool-1-thread-1] 17:13:14,450 repast.simphony.batch.ssh.SessionsDriver - Finished. Elapsed Time: 1.1058
>
>
> I have already tried changing the boolean parameters to String within parameters.xml and then parsing them as booleans within the model code.  That eliminated the first set of errors, but I still got the second sting of errors and the runs failed again.
>
> The batch runs previously worked in Repast 2.3.1, and I’m now using 2.4.0.  I had made a number of edits in between the time I updated to 2.4.0 and when I tried do batch runs.  I have not, however, made any changes since that time to parameters.xml or the code that reads in the parameters.
>
> Any help would be appreciated.  I’m hoping not to have to go back to 2.3.1 because I’m worried that will entail a lot of work (and time).
>
> Thank you,
>
> Mike
>
>
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Repast-interest mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/repast-interest


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Repast-interest mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/repast-interest
Loading...