Ticket #4120 (closed support: fixed)

Opened 7 years ago

Last modified 5 years ago

Migrate from mingw builds to NATIVE Windows (cl.exe compiler) builds

Reported by: wjbohnh Owned by: wjbohnh
Priority: high Milestone:
Component: Porting/Distribution Version: VOTD
Severity: normal Keywords:
Cc: briadam, dmvigi, msebeid Due Date:
Parent Tickets: #4025 Blocked By:
Blocking: Requestor:
Sensitive: no

Description (last modified by wjbohnh) (diff)

We need to make this a higher priority task now that we depend on Boost libs (binary components) which are NOT supported on MINGW:

 http://www.boost.org/doc/libs/1_45_0/more/getting_started/windows.html

Subtickets

#4182: Apply patches for Microsoft Visual Studio portabilitytaskclosedwjbohnh

Attachments

viewNotes.htm Download (8.9 KB) - added by wjbohnh 6 years ago.
CMake cache initialization file used by Kitware for VisualStudio? builds
compilervas.bat Download (3.4 KB) - added by wjbohnh 6 years ago.

Change History

comment:1 Changed 7 years ago by wjbohnh

Just to clarify, Boost is NOT supported on MinGW:

 http://www.boost.org/doc/libs/1_45_0/more/getting_started/windows.html

Therefore, in my opinion, we are setting ourselves up for failure by providing mingw binaries when the foundation binary libs (Boost) are NOT supported (the "may or may not work" comment is silly -- we WILL eventually hits bugs). Note that Boost IS supported on Cygwin, and should therefore be emphasized as the best option for Windows customers (until we fully support NATIVE Windows, DevStudio?/MS compiler (cl.exe) builds).

I hope that clarifies my position with respect to Windows builds and delivery of Windows binaries to users.

comment:2 Changed 7 years ago by wjbohnh

  • Cc briadam, dmvigi added; briadam@…, dmvigi@… removed
  • Status changed from new to accepted
  • Description modified (diff)

comment:3 Changed 7 years ago by wjbohnh

Hopefully, a proper MSVC configuration for JEGA is easy; my gut tells me that it should be a fairly minor mod to ensure that settings like “EDDY_HAVE_INT32_T” are defined properly, and/or (possibly) including a (windows?) header file to ensure desirable typedefs, such as:

typedef int32 int32_t;

comment:4 follow-up: ↓ 5 Changed 6 years ago by wjbohnh

  • Cc msebeid added

Need to attach the cache initialization file that is being used by Kitware for the nightly (PUBLIC) Dakota MSVC/ifort build/test:
http://dakota.sandia.gov/cdash/viewNotes.php?buildid=847

I will NOTE that Bill Hoffman (which CUBIT team also ass) has always recommended re-packaging fortran symbols into separate: (A) libraries (DONE: #4182) and
(B) directories (not done),
and information from CUBIT team seems to support these strategies.

HOWEVER, Bill Hoffman has indicated that (B) is NOT a must do, UNLESS gfortran is supported. Therefore, I want to ensure that Mohamed is using ifort!

Finally, the platform setup URL might be handy for future ref:
 https://software.sandia.gov/trac/dakota/wiki/PlatformSpecificSetup

Changed 6 years ago by wjbohnh

CMake cache initialization file used by Kitware for VisualStudio? builds

comment:5 in reply to: ↑ 4 Changed 6 years ago by wjbohnh

Replying to wjbohnh:

I will NOTE that Bill Hoffman (which CUBIT team also ass) has always recommended ...

Meant to say CUBIT team also ASSERTS.. To bad I did not finish my train of thought there!

comment:6 Changed 6 years ago by wjbohnh

briadam wrote:

Aday currently has:

. CMake 2.8.9 (on system path for all users)
. MSVS 9 (2008)
. Intel Visual Fortran 11.1
. LAPACK 3.4.2, built with CMake and ifort (static libraries, 32-bit MSVS 2008)
. C:\projects\dakota\utils\lapack-3.4.2\build
. BoostPro 1.47, for MSVS 9, multithreaded and single threaded for static runtime,
    no debug, some files didn’t install correctly, but I think it working
    C:\Program Files (x86)\boost\boost_1_47

wjbohnh will update the wiki for Windows Developers based on his Aday "quick start" experience.

Changed 6 years ago by wjbohnh

comment:7 Changed 6 years ago by wjbohnh

On aday.sandia.gov, to setup (set environment variables) a dev environment for intel (ifort) and MKL (for linalg), need to call a bat script:

'C:\Program Files (x86)\Intel\Compiler\11.1\070\bin\ifortvars.bat'

(I will also attach the version I've been using on s843487 for future ref/comparison)

comment:8 Changed 6 years ago by Rakushun

Dear Dakota team,

I successfully managed to compile and link Dakota using Visual Studio 2005 in Windows 7. It runs nearly out of the box, provided that you take some care at the one or the other situation. Here a small overview about the steps I did in hope, it can help someone else who tries it.

Dakota-version: Current SVN checkout (~5.2+)
CMake-Version: 2.8.10.2
Lapack-Version: 3.4.2
Boost-Version: 1.52.0
VS-Version: Visual Studio 8 (2005)
Fortran-Compiler: Intel Fortran 11.1
Windows version: Windows 7

1.) Create a directory for all projects, let's say: "C:\opt"

2.) Download CMake and install it.

3.) Download Lapack, unzip it to "C:\opt\LAPACK". Create a target directory "C:\opt\LAPACK_BUILD".

4.) Download Boost, unzip it to "C:\opt\BOOST".

5.) Check out the current DAKOTA branch to "C:\opt\DAKOTA". Create a target directory "C:\opt\DAKOTA_BUILD". You will need the SVN checkout, as the published sourcecode of version 5.2 is not Visual Studio compatible due to improper CMake-files.

6.) Start a command shell with environment settings prepared for the Intel Fortran compiler -- i.e., start the "ifortvars.bat" via the start menu of Windows.

IMPORTANT: START THE COMMAND SHELL IN "ADMINISTRATOR" MODE VIA RIGHT-CLICK ON IT. We have to work from this command shell from now on, otherwise, the Intel compiler will create libraries with symbols in lower-case which may not be correctly linked to DAKOTA. We will need the Administrator mode for correct project files for Visual Studio 2005.

7.) In the command shell, manually start the CMake tool: Just right-click on the CMake symbol in the start menu, select "properties", copy the link pointing at CMake and paste it to the command shell. This will start CMake in Admin-Mode with environment settings appropriate for the Intel Fortran compiler. Do not close the command shell, we need it later.

8.) In CMake, select "C:\opt\LAPACK" as input and "C:\opt\LAPACK_BUILD" as output directories. Click on "Generate" to generate the CMake variables. Click on "Build" and select "Visual Studio 8 (2005)" to create the project files for Visual Studio. Exit CMake for the moment.

9.) Open Visual Studio 2005 in Administrator mode (by right-click on it). Open the project files in "C:\opt\LAPACK_BUILD" and build the complete project. This will give you the BLAS and LAPACK library in "C:\opt\LAPACK_BUILD\lib".

10.) Using the still open console, switch the directory to "C:\opt\BOOST" (e.g. by "cd /D C:\opt\BOOST") and build the BOOST library as static library. In version 1.52.0, this is done by executing the commands "bootstrap" and ".\b2" in this directory. As we are in a command shell with the environment defined for the IFC/Visual Studio environment, the correct compiler settings should be discovered automatically. The compilation really takes a while, but at the end, you can find a couple of compiled libraries in "C:\opt\BOOST\stage\lib".

11.) In the command shell, start the CMake tool again, see 7.)

12.) In CMake, select "C:\opt\DAKOTA" as input and "C:\opt\DAKOTA_BUILD" as output directories. Click on "Generate" to generate the CMake variables -- this will fail, which is ok for the moment. Afterwards, change the following settings in the CMake-variables:

  • Set "BOOST_ROOT=C:\opt\BOOST" (create if it does not exist)
  • Set "LAPACK_DIR=C:\opt\LAPACK_BUILD"
  • Set "BLAS_LIB=C:\opt\LAPACK_BUILD\lib\Debug\blas.lib" or "BLAS_LIB=C:\opt\LAPACK_BUILD\lib\Release\blas.lib" (depending on whether you build in debug or release mode).
  • Set "BUILD_SHARED_LIBS=off" to switch off the use of shared libraries in DAKOTA -- otherwise, the static BOOST libraries are not detected correctly.

13.) Click on "Generate" to update the CMake settings, this should work now. Assure that CMake detected the correct LAPACK build directory, it should have set "LAPACK_DIR=C:\opt\LAPACK_BUILD". Click on "Build" and select "Visual Studio 8 (2005)" to create the project files for Visual Studio. Exit CMake and close the command shell, we are done here.

14.) Open Visual Studio 2005 in Administrator mode (by right-click on it). Open the project files in "C:\opt\DAKOTA_BUILD" and build the complete project (which takes a while). This should give you a working DAKOTA executable, linked to native BLAS, LAPACK and BOOST libraries.

15.) Right-click on the subproject "dakota" and "Select this as start project". Now, by hitting "F5", you can start it in debug mode or whatever you want. In order to run the test examples, you will have to define the command line parameters and the execution directory manually. You should remove the subprojects "test_dakota" and "test_lib_dakota" as they want to be compiled on every execution, although there is nothing to compile. These projects are not needed.

That's it. As you can see, it nearly works out-of-the box, and I can imagine, for Visual Studio 2008 and higher, you will most likely not need the administrator mode -- for Visual Studio 2005 however, it is mandatory, since this software (like Visual Studio 2003) is not completely Win-7 compatible otherwise. The essential point is that CMake is started from the IFC command shell in all situations because otherwise, it does not properly detect and configure the IFC compiler needed for BLAS, LAPACK and DAKOTA.

To compile with MPI-support:

1.) Install MPICH-2. Activate the SMTP service and register your account by typing

smpd -install
mpiexec -remove
mpiexec -register
mpiexec -validate
smpd -status

2.) Open the CMake GUI from the administrator-privileged IFC command line (see above).

3.) Set "DAKOTA_HAVE_MPI = true".

4.) Set "MPI_INCLUDE_PATH" = [MPI installation Directory]\include

5.) Set "MPI_LIBRARY" = [MPI installation Directory]\include

6.) Set "MPI_BASE_DIR" = [MPI installation Directory]

7.) Set "MPIEXEC=[MPI installation Directory]\mpiexec.exe"

8.) In the project file, change the settings of the project "dakota_src" and add the path to "[MPI installation]\include" to the include paths.

Click on "Configure" and "Generate", this will create the appropriate makefiles.

To debug MPI applications with Visual Studio:

1.) Install the "Remote Debugger" using the setup of Visual Studio.

2.) Remember the path to the file "mpishim.exe". This file is installed with the Remote Debugger and can usually be found in the folder
C:\Program Files (x86)\Microsoft Visual Studio 8\Common7\IDE\Remote Debugguer\x86.

3.) Open the DAKOTA project file and change the options of the subproject "dakota" (which is used as start project).

  • Define 'MPI Cluster Debugger' as debugger to use
  • As 'MPIShim-Location', insert name+path to the MPISHIM.EXE file from above.
  • As 'MPIRun' command, insert name + path to the MPIEXEC.EXE file which was installed with MPICH-2.
  • As parameters for MPIExec, insert the usual parameters, e.g. -np 2 to use 2 processors.

Now you should be able to compile and run the application with MPI.

Ok, I hope I didd not forget anything :-)

Regards,

Michael


P.S.: It is necessary to modify the CMake files of dakota_src/dakota in order to respect the values in MPI_INCLUDE_PATH and MPI_LIBRARY, I posted that earlier.

comment:9 Changed 5 years ago by ycollette.nospam@…

Hello,

I am currently building dakota under windows XP 64 + Visual studio 2010 and intel compilers 11.
The file DakotaBuildInfo?.cpp doesn't seems to be generated.
I use an export of the dakota svn. So, this is not a svn repo. And so, the revision number can't be extracted via the cmake command.

YC

comment:10 Changed 5 years ago by wjbohnh

  • Status changed from accepted to closed
  • Resolution set to fixed

Closing ticket since the "port" is complete. Any issues that come up with respect to native Windows builds should have new tickets created.

Note: See TracTickets for help on using tickets.