Comments you submit will be routed for moderation. If you have an account, please log in first.
Modify

Ticket #4069 (closed enhancement: fixed)

Opened 3 years ago

Last modified 18 months ago

Scilab interfaces for Dakota

Reported by: ycollette.nospam@… Owned by: briadam
Priority: low Milestone: 5.3
Component: Software Requirements - User Version:
Severity: normal Keywords:
Cc: wjbohnh, yann.collette@… Due Date:
Parent Tickets: Blocked By:
Blocking: Requestor:
Sensitive: no

Description

Three Scilab interfaces have been implemented :

  • script interface
  • compiled interface
  • linked interface.

I joined some patched for those interfaces.

script interface : copy dakota_scilab_script.patch in /dakota/trunk/examples/script_interfaces/Scilab/ and run patch -p0 < dakota_scilab_script.patch

compiled interface : copy dakota_scilab_compiled.patch in /dakota/trunk/examples/compiled_interfaces/Scilab/ and run patch -p0 < dakota_scilab_compiled.patch

linked interface :

  • copy dakota_scilab_linked.patch in /dakota/trunk/examples/linked_interfaces/Scilab/ and run patch -p0 < dakota_scilab_linked.patch
  • copy dakota_scilab_src.patch in /dakota/trunk/src/ and run patch -p0 < dakota_scilab_src.patch

Latex doc : copy dakota_scilab_latex.patch in /dakota/trunk/docs/latex-user/ and run patch -p0 < dakota_scilab_latex.patch

To build with cmake : copy dakota_scilab.patch in /dakota/trunk and run patch -p0 < dakota_scilab.patch

Subtickets

Attachments

dakota_scilab_script.patch Download (38.1 KB) - added by anonymous 3 years ago.
dakota_scilab_compiled.patch Download (2.4 MB) - added by anonymous 3 years ago.
dakota_scilab_linked.patch Download (179.6 KB) - added by anonymous 3 years ago.
dakota_scilab_src.patch Download (25.4 KB) - added by anonymous 3 years ago.
dakota_scilab_latex.patch Download (9.4 KB) - added by anonymous 3 years ago.
dakota_scilab.patch Download (8.0 KB) - added by anonymous 3 years ago.
FindScilab.zip Download (1.7 KB) - added by Collette Yann <yann.collette@…> 3 years ago.
a find package to detect scilab with cmake

Change History

Changed 3 years ago by anonymous

Changed 3 years ago by anonymous

Changed 3 years ago by anonymous

Changed 3 years ago by anonymous

Changed 3 years ago by anonymous

Changed 3 years ago by anonymous

comment:1 Changed 3 years ago by briadam

  • Reporter changed from anonymous to yann.collette@…

Changed 3 years ago by Collette Yann <yann.collette@…>

a find package to detect scilab with cmake

comment:2 Changed 3 years ago by slefant

  • Owner changed from dakota-developers@… to slefant
  • Status changed from new to accepted

comment:3 Changed 3 years ago by slefant

  • Cc wjbohnh added
  • Owner changed from slefant to briadam
  • Status changed from accepted to assigned

comment:4 Changed 3 years ago by anonymous

Hello,

Do you plan to add the scilab interface in the next dakota release ?

Best regards,

YC

comment:5 Changed 3 years ago by briadam

Yann -- We didn't have the cycles to review and include before the 5.2 release. However, I now have the necessary information on what consent I'll need from you and any other authors to be able to accept the contribution. With the release behind, can send to you soon (hopefully later today once I collect up my notes on the subject).

Incidentally, we also hope to do a consolidation of the Matlab, SciLab?, and Python interfaces to reduce maintenance burden. Once we integrate the SciLab? example, we'll take a pass at that. -- Brian

comment:6 Changed 3 years ago by briadam

  • Type changed from support to enhancement
  • Component changed from Support to Software Requirements - User

I sent separate email to Yann requesting authorship and license information.

comment:7 Changed 2 years ago by briadam

  • Priority changed from normal to low
  • Milestone set to 5.3

We received all permission necessary to integrate this capability. Scheduling for next release review.

comment:8 Changed 2 years ago by briadam

  • Cc yann.collette@… added
  • Reporter changed from yann.collette@… to ycollette.nospam@…

Updating reporter and cc. Beginning integration work on this.

comment:9 Changed 2 years ago by briadam

Referenced in changeset [1086]:

Initial commit of contributed direct interface to Scilab. Initial pass includes the changes to the build system and the dakota/src directory. This needs considerable work, but I want to collaborate through the svn repo. Remaining issues are being tracked in Ticket #4069.

comment:10 Changed 2 years ago by briadam

Yann,

I dug in on this (at long last!) as we have some other pending refactoring work to do in the direct interfaces section.

I tested against Scilab 5.3.3. I reviewed and started to integrate:

  • Script interface (dakota_scilab_script.patch): seems to work, though some data files are missing; we'll probably just share them among the script and linked examples...
  • User's manual update (dakota_scilab_latex.patch): good summary
  • Direct interface (dakota_scilab_src.patch): compiles with small changes
  • CMake system (dakota_scilab.patch, FindScilab): minor edits, working

Main problem: Before I started editing I was able to get your files to compile with minimal changes. However I get this run error:

  • Dakota_out vs. Dakota
    Direct function: invoking scilab 
    Error (Direct:Scilab): Failed to get variable Dakota_out from Scilab.
    API Error:
    	in readNamedList: Unable to get variable "Dakota_out"
    	in getVarAddressFromName: Unable to get address of variable "Dakota_out"
    Failure captured: aborting...
    

Any idea how to address? I tried allocating memory for that structure on the DAKOTA/C++ side and also just re-using the Scilab variable Dakota as the return argument. No luck.

Along the way I tried to do the following:

  • Cleanup for style
  • Turned debugging comments into verbosity-controlled prints
  • Reduced copies where seems possible
  • Removed a few unused variables
  • Fixed memory leaks where was being malloced again without free

General Questions:

  • I was able to add LINK_OPTS to the CMake LINK_FLAGS for the DAKOTA executables. Is that what you did in your build?
  • It seems like Scilab supports matrices of integers. Should those be used to receive DAKOTA integers and integer arrays? Or best to leave all as doubles like you have? I debated about the same in creating the Matlab interface, but the Matlab APIs for creating matrices of ints seemed weaker at the time... If we passed as integers we could eliminate some more data copies.
  • When creating a Scilab matrix, it appears that Scilab copies the data from the passed pointer. Due to this I eliminated a lot of intermediate data copies, but am not sure if that's correct.
  • In your development work, why didn't new/delete work? Why did you have to change to malloc/free?


I committed my current version of the code to DAKOTA trunk. You should be able to check it out of the repo.

comment:11 Changed 2 years ago by ycollet

For the execution problem, I will dig and I will send an answer later.
My cmake command:

cmake -DCMAKE_INSTALL_PREFIX=/opt/dakota-devel -DDAKOTA_SCILAB=ON -DSCILAB_USER_PATH:Path=/opt/scilab-master ..

General Questions:

  • I was able to add LINK_OPTS to the CMake LINK_FLAGS for the DAKOTA executables. Is that what you did in your build?

On my side, I modified the src/CMakeLists.txt file and a .C file:

Index: CMakeLists.txt
===================================================================
--- CMakeLists.txt (révision 1086)
+++ CMakeLists.txt (copie de travail)
@@ -211,7 +211,8 @@

set(SCI_LINK_FLAGS "

TracMath macro processor has detected an error. Please fix the problem before continuing.
Problem creating tex file: [Errno 13] Permission denied: u'/etc/trac/TracMath_cache/196759077f9c29f8763628cb7e33004fd67b4aed.tex'
{SCILAB_ROOT}/lib/scilab")
set(SCI_LINK_FLAGS "
TracMath macro processor has detected an error. Please fix the problem before continuing.
Problem creating tex file: [Errno 13] Permission denied: u'/etc/trac/TracMath_cache/dbae3af22ba7ce6b5af275f7b0e48d03dd9c0397.tex'
{SCILAB_ROOT}/lib/thirdparty")
set(SCI_LINK_FLAGS "
TracMath macro processor has detected an error. Please fix the problem before continuing.
Problem creating tex file: [Errno 13] Permission denied: u'/etc/trac/TracMath_cache/196759077f9c29f8763628cb7e33004fd67b4aed.tex'
{SCILAB_ROOT}/lib/thirdparty")

+ set(SCI_LINK_FLAGS "${SCI_LINK_FLAGS} -L/usr/lib64 -L/usr/lib -lgfortran")
+

if (EXISTS ${SCILAB_ROOT}/thirdparty/java/lib/amd64)

set(SCI_ARCH "amd64")

else()

Index: DakotaInterface?.C
===================================================================
--- DakotaInterface?.C (révision 1086)
+++ DakotaInterface?.C (copie de travail)
@@ -21,6 +21,7 @@

#include "GridApplicInterface?.H"
#endif DAKOTA_GRID
#include "DirectApplicInterface?.H"

+#undef round

#include "ApproximationInterface?.H"
#ifdef HAVE_AMPL
#undef NO avoid name collision from UTILIB

  • It seems like Scilab supports matrices of integers. Should those be used to receive DAKOTA integers and integer arrays? Or best to leave all as doubles like you have? I debated about the same in creating the Matlab interface, but the Matlab APIs for creating matrices of ints seemed weaker at the time… If we passed as integers we could eliminate some more data copies.

Yes, Scilab supports matrix of integers. But these integers are used for special cases. Normally, indexes and other type like these are stored in scilab as double (normally, like in matlab). Matlab and Scilab have the same history. They use to have the same source code years ago when matlab was "open source" :)

  • When creating a Scilab matrix, it appears that Scilab copies the data from the passed pointer. Due to this I eliminated a lot of intermediate data copies, but am not sure if that's correct.


Every Scilab API function which start with "create" copy the data passed as a parameter. Otherwise, with the function which start with "alloc", you first create an memory space into scilab and then, you need to transfert the data from your pointed area to the pointed area allocated by scilab.

  • In your development work, why didn't new/delete work? Why did you have to change to malloc/free?

I used normally MALLOC and FREE (capitalized letters). These functions are shipped by Scilab and should be used to allocate

comment:12 Changed 2 years ago by ycollet

To be able to launch the example from examples/linked_interfaces/scilab, I performed the following steps:

export LD_PRELOAD=/usr/lib64/libstdc++.so.6:/usr/lib64/libgfortran.so.3 # Because I installed the binary version downloaded from the scilab website.

export DAKOTA_SCILAB_STARTUP="exec rosenbrock.sci;" # To load into scilab the rosenbrock.sci function

~/repositories/dakota/build/src/dakota dakota_rosenbrock_scilab.in > log.txt

And the optimization went fine.

I added a new change in the interface: (to be able to launch scilab command after the call to the objective function - for debugging purpose for example).
Here is the patch against the current trunk.

--- DirectApplicInterface?.C (révision 1086)
+++ DirectApplicInterface?.C (copie de travail)
@@ -4561,6 +4561,9 @@

testJob = SendScilabJob?((char *)analysis_command.c_str());


+ char * env_scilab_post_startup = getenv("DAKOTA_POST_SCILAB_STARTUP");
+ if (env_scilab_post_startup) SendScilabJob?(env_scilab_post_startup);
+

if (testJob == -1) {

Cerr << "Error (Direct:Scilab): Call Scilab has not been able to write "

<< "the job into Scilab." << std::endl;

comment:13 Changed 2 years ago by briadam

Via email to briadam 20120730:

Hello,

Here is a comment which has been rejected by trac because it has considered my comment as spam (to many external comment by ticket):

I forgot to add something:

I you use the scilab binary distribution downloaded from the website, you will need to set another environment variable: SCI

export SCI=<Where_I_installed_scilab>/share/scilab

For the toolbox used to call foreign languages: actually only the java toolbox is available in atoms (ATOMS is a toolbox which allows to install automatically some toolboxes). Here is the link:

 http://atoms.scilab.org/toolboxes/JIMS

For the other languages (C# and python), the code is actually on the scilab forge:

 http://forge.scilab.org/index.php/p/dnims/ # for C#  http://forge.scilab.org/index.php/p/pims/ # for Python  http://forge.scilab.org/index.php/p/JIMS/ # For Java

You can have a look in the unit tests to see how to use these toolboxes:

 http://forge.scilab.org/index.php/p/dnims/source/tree/master/test/unit_tests
 http://forge.scilab.org/index.php/p/pims/source/tree/master/tests/unit_tests
 http://forge.scilab.org/index.php/p/JIMS/source/tree/master/test/unit_tests

Best regards,

YC

comment:14 Changed 18 months ago by briadam

Referenced in changeset [1327]:

Laura: I commented out your likelihood function as I couldn't get it to compile.

Add Scilab documentation, contrbiuted by Yann Collette. Ticket #4069.

Adjust discussion of Matlab data structures. More to do there.

Minor tweaks to discussion of least squares in Tutorial.

comment:15 Changed 18 months ago by briadam

I didn't realize I was missing the script example. I committed both that (changeset:1332) and the compiled one (changeset:1333), though the latter is likely to vanish in future development. This reveals a need to reorganize our examples directory, so the Scilab script and data files can be shared among the various interface variants.

Yann: Could you please confirm that there are no CeCILL or GPL-licensed files in any of the examples? Particularly the automatic_test.xcos and Bridge/macros files? Also, I get an error about localise2d in FemTruss?. Perhaps we just instruct folks to download your FemTruss? package if needed? I added the docs to the User's manual, after editing.

comment:16 Changed 18 months ago by briadam

Referenced in changeset [1337]:

Adding README describing how to obtain Scilab/femtruss. This closes out Ticket #4069.

comment:17 Changed 18 months ago by briadam

  • Status changed from assigned to closed
  • Resolution set to fixed

All licensing issues are resolved. Added instructions for getting FemTruss? package to the examples directories.

View

Add a comment

Modify Ticket

Action
as closed
The resolution will be deleted. Next status will be 'reopened'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.