Posts by author wehart

PyUtilib 5.1.3548 Release

We are pleased to announce the release of PyUtilib 5.1 (5.1.3548). The PyUtilib project supports the development of an ensemble of Python packages that include a wide variety of utilities, including a well-developed component architecture.

The following are highlights of this release:

  • Adding multiple task submission capability to Pyro dispatcher.
  • Adding support for xlrd and openpyxl interfaces.
  • Changing the semantics of Config* objects to return the values of ConfigValue objects.
  • When loading dictionaries, recursively create Container objects for sub-dictionaries.
  • Updates to the ConfigBlock
    • Support for multiple argument definitions for a single ConfigValue
    • Support for declaring groups by passing argparse parser or group objects

See https://software.sandia.gov/trac/pyutilib/wiki/GettingStarted for instructions for getting started with PyUtilib. PyUtilib can be downloaded from PyPI.

Enjoy!

PyUtilib 5.0.3478 Release

We are pleased to announce the release of PyUtilib 5.0 (5.0.3478). The PyUtilib project supports the development of an ensemble of Python packages that include a wide variety of utilities, including a well- developed component architecture.

The following are highlights of this release:

  • Integrating many PyUtilib sub-packages into a single source tree
  • Integrating the Coopr plugin framework in pyutilib.component.core
  • pyutilib.th
    • Adding a forcedskip option, which can be used to add tests that are skipped due to external factors.
    • Extending baseline tests to allow comparison with YAML, XML and JSON files.
  • pyutilib.pyro
    • Changes to support clearing the dispatcher queue
    • Allowing more control of blocking behavior when calling dispatcher
    • Adding randomization of the sleep interval in a pyro client, following failure to initially connect to a name server.
  • pyutilib.misc
    • Allowing None tolerance when comparing YAML files.
    • Adding support for gzip and bz2 compressed files, which act as single file archives
    • Adding support to open readonly file objects (using the open() method) rather than only supporting file extractions to disk

See https://software.sandia.gov/trac/pyutilib/wiki/GettingStarted for instructions for getting started with PyUtilib. PyUtilib can be downloaded from PyPI.

Enjoy!

PyUtilib 4.1.2882 Release

We are pleased to announce the release of PyUtilib 4.1 (4.1.2882). The PyUtilib project supports the development of an ensemble of Python packages that include a wide variety of utilities, including a well-developed component architecture.

The following are highlights of this release:

  • Added the functor_api capability (in pyutilib.workflow)
  • The TempfileManager utility now supports nested tempfile management.

See https://software.sandia.gov/trac/pyutilib/wiki/GettingStarted for instructions for getting started with PyUtilib. PyUtilib can be downloaded from PyPI.

Enjoy!

  • Posted: 2012-07-31 21:16 (Updated: 2012-07-31 21:17)
  • Author: wehart
  • Categories: (none)
  • Comments (0)

PyUtilib 4.0 Release

We are pleased to announce the release of PyUtilib 4.0 (4.0.2848). The PyUtilib project supports the development of an ensemble of Python packages that include a wide variety of utilities, including a well-developed component architecture.

The following are highlights of this release:

  • Compatibility changes to support Python 3.x (Support for Python 3.2 is confirmed; Python 3.1 still causes some problems)
  • Many minor bug fixes that were highlighted in the 3.x conversion.

See https://software.sandia.gov/trac/pyutilib/wiki/GettingStarted for instructions for getting started with PyUtilib. PyUtilib can be downloaded from PyPI, and installers are available for MS Windows and Unix operating systems.

Enjoy!

PyUtilib 3.8.2652 Release

We are pleased to announce the release of PyUtilib 3.8 (3.8.2652). The PyUtilib project supports the development of an ensemble of Python packages that include a wide variety of utilities, including a well-developed component architecture.

The following are highlights of this release:

  • Scalability improvements for Pyro parallelism.
  • Update to support the (guided) automatic creation of branches in pyutilib.svn
  • Defer the reading of config files until after ZIP files are opened up in a pyutilib virtualenv installer.

This includes updates to the following packages:

  • pyutilib.misc 5.0.1
  • pyutilib.pyro 3.5
  • pyutilib.svn 1.1
  • pyutilib.virtualenv 2.7

See https://software.sandia.gov/trac/pyutilib/wiki/GettingStarted for instructions for getting started with PyUtilib. PyUtilib can be downloaded from PyPI, and installers are available for MS Windows and Unix operating systems.

Enjoy!

PyUtilib 3.6.2498 Release

We are pleased to announce the release of PyUtilib 3.6 (3.6.2498). The PyUtilib project supports the development of an ensemble of Python packages that include a wide variety of utilities, including a well-developed component architecture.

The following are highlights of this release:

  • Support for Python 2.4 is deprecated.
  • Consistent reformatting using reindent
  • Major changes to capabilities in pyutilib.th
    • Added two new nose plugins
    • API changes for dynamic test logic
  • Added the create_setup script in pyutilib.virtualenv to manage the creation of Windows installers

This includes updates to the following packages:

  • pyutilib.autotest 1.4.6
  • pyutilib.common 3.0.5
  • pyutilib.component.app 3.1.6
  • pyutilib.component.config 3.3.2
  • pyutilib.component.core 4.3.1
  • pyutilib.component.executables 3.4.1 -pyutilib.component.loader 3.3.1
  • pyutilib.dev 1.15.2
  • pyutilib.enum 1.0.7
  • pyutilib.excel 3.0.6
  • pyutilib.math 3.0.6
  • pyutilib.misc 4.6
  • pyutilib.ply 3.0.6
  • pyutilib.pyro 3.3
  • pyutilib.R 3.0.5
  • pyutilib.services 3.3.1
  • pyutilib.subprocess 3.2.3
  • pyutilib.svn 1.0.1
  • pyutilib.th 5.0
  • pyutilib.virtualenv 2.5
  • pyutilib.workflow 2.2.4

See https://software.sandia.gov/trac/pyutilib/wiki/GettingStarted for instructions for getting started with PyUtilib. PyUtilib can be downloaded from PyPI, and installers are available for MS Windows and Unix operating systems.

Enjoy!

Support for Python 2.4 is being deprecated

In the next release, PyUtilib will no longer support portability to Python 2.4. We are starting to see PyUtilib test failures due to changes in third-party packages that no longer support Python 2.4. Since there are no immediate drivers for Python 2.4 support for PyUtilib, this version is being dropped from PyUtilib tests.

  • Posted: 2011-05-21 10:55 (Updated: 2011-05-21 10:55)
  • Author: wehart
  • Categories: News
  • Comments (0)

PyUtilib 3.5.2320 Release

Another minor release of PyUtilib has been finalized. This includes updates to the following packages:

pyutilib.dev 1.15 
pyutilib.misc 4.5.3 
pyutilib.virtualenv 2.4.1 

Some highlights of this release include

  • Adding some diagnostic output for the Python executable used to create the virtual Python environment.
  • Added a slightly more robust importer. If the filename ends with '.py', then don't try to import it.

PyUtilib 3.5.2313 Release

Another minor release of PyUtilib has been finalized. This includes updates to the following packages:

pyutilib.misc 4.5.2
pyutilib.ply 3.0.5
pyutilib.virtualenv 2.4

Some highlights of this release include

  • Fixes in the pypi_downloads script to estimate pypi downloads between given start and end times
  • Many fixes to the pyutilib.virtualenv bootstrap logic to resolve installation issues under MS Windows

PyUtilib 3.4.2289 Release

Another minor release of PyUtilib has been finalized. This includes updates to the following packages:

pyutilib.autotest 1.4.4
pyutilib.component.app 3.1.5
pyutilib.component.config 3.3.1
pyutilib.component.core 4.3
pyutilib.component.loader 3.3
pyutilib.dev 1.14
pyutilib.enum 1.0.6
pyutilib.excel 3.0.5
pyutilib.math 3.0.5
pyutilib.misc 4.5.1
pyutilib.pyro 3.2.1
pyutilib.th 4.6
pyutilib.virtualenv 2.3
pyutilib.workflow 2.2.2

Some highlights of this release include

  • Changes to make installation with vpy_install more robust
  • Changes in test syntax to use the assert*() syntax for test methods
  • Overhaul of LogHandler class, and cleanup of StreamIndenter
  • Added PauseGC utility for cleanly pausing the Python garbage collector.
  • Consistently use the Python logging utility to manage diagnostic output

PyUtilib 3.3.2082 Release

Another minor release of PyUtilib has been finalized. This includes updates to the following packages:

 pyutilib.autotest 1.4.1
 pyutilib.component.executables 3.4
 pyutilib.misc 4.2
 pyutilib.services 3.3
 pyutilib.subprocess 3.2.2
 pyutilib.th 4.4
 pyutilib.workflow 2.2

Some highlights of this release include

  • Allowing executable service to apply a filter function
  • Fixed subprocess spawning bug on MS Windows
  • Extended baseline test syntax to allow the specification of a working directory and to create a bash shell script that executes the command
  • Fixes for the setup of workflow drivers

PyUtilib 3.2.2053 Released!

Another minor release of PyUtilib has been finalized. This includes updates to the following packages:

- Minor updates
    pyutilib.dev 1.11.1
    pyutilib.th 4.3.1
    pyutilib.workflow 2.1
- Revision updates
    pyutilib.component.app 3.1.3
    pyutilib.component.config 3.2.3
    pyutilib.component.loader 3.2.3
    pyutilib.misc 4.1.2
    pyutilib.virtualenv 1.5.1

Some highlights of this release include

  • Change of test.pyutilib semantics to perform smoke tests by default
  • Portability fixes for Python 2.7
  • Adding workflow logic to support branching
  • Adding support for testing categories

New Documentation for pyutilib.workflow

I've created new documentation for pyutilib.workflow here. The API for defining workflows is fairly mature now. I'll probably only work on it to the extent that applications require extensions. In particular, I have not released this version of pyutilib.workflow since we plan to use it in some new development shortly ... so there may be changes and bug fixes.

FWIW, I have a longer-term plan to support export of workflows to formats that can be executed with grid/cloud computing frameworks (e.g. DAGMAN), and to try managing parallelism in workflows with RPC systems (e.g. pyro).

Additionally, I've stuck with LaTeX documentation because I know this well. I've considered moving to Sphinx, which is now standard for Python packages. However, I don't know of an easy way to publish Sphinx documentation within Trac.

Note to developers: Subversion post-commit hook

John Siirola has customized the subversion post-commit script to automatically annotate Trac tickets with information from subversion commit messages.

The script looks for strings that match '(?:#|(?:ticket|issue|bug)[: ]?)\d+' . For example:

  #123
  ticket 123
  ticket:123
  issue 123
  issue:123
  bug 123
  bug:123

When this regular expression is matched, the associated Trac ticket will be automatically updated to include the commit message to the ticket discussion.

New Package: pyutilib.autotest

The pyutilib.autotest package provides a facility for automatically configuring tests that are executed with Python's unittest package. This capability is tailored for tests where one or more solvers are applied to one or more problems. This testing structure is particularly useful for evaluating the execution of external executables on datasets.

There are three main steps for configuring and applying pyutilib.autotest:

  1. Create a configuration file
  2. Create a test driver plugin
  3. Setup the Python module that will be used to apply the tests

A detailed description of how these steps is provided in the  pyutilib.autotest PyPI documentation.

This package is being used to simplify testing in Coopr and Acro.

  • Posted: 2010-06-27 23:31
  • Author: wehart
  • Categories: (none)
  • Comments (0)

A new version scheme for PyUtilib

One of the difficulties for managing a project like PyUtilib is that there are a lot of independent sub-packages in this project. Still, we'd like to occasionally say that the entire 'project' has a release.

To do this, I've created the pyutilib package, which is registered in PyPI as PyUtilib. This package is nothing more than a Python package that requires other PyUtilib packages to be installed. However, the challenge is that the versions on these packages may be inconsistent.

To address this, we've settled on the following version scheme for PyUtilib:

  <major>.<minor>.<revision>

where the revision number comes from subversion. This scheme has the typical semantics:

  • Changes in major revisions denote updates that may not be backwards compatible
  • Changes in minor revisions denote updates in capabilities that should be backwards compatible
  • Changes in the revision reflect patch updates to existing capabilities

The use of revision numbers here is intentional. It provides a clean semantics for defining a PyUtilib release: it contains the latest releases of all packages with the corresponding revision number.

When releases occur with new major or minor numbers, we can generally refer to them by version <major>.<minor> . Referring to the revision number is really only necessary when distinguishing between different patch releases.

  • Posted: 2010-04-14 16:17
  • Author: wehart
  • Categories: (none)
  • Comments (0)

Release 3.0

I have just finished processing the PyUtilib 3.0 release. This release reflects several changes in PyUtilib:

  • PyUtilib has been decomposed into separate projects, and PyUtilib 3.0 is a release of a collection of PyUtilib projects.
  • PyUtilib is now a separate project that is registered with PyPI. If you use easy_install or pip to install PyUtilib, it ensures that an ensemble of packages are installed.
  • We've settled on a versioning scheme for PyUtilib. This is the PyUtilib 3.0.1745 release. I'll blog more about this in a moment.
  • We've finished a design document for the PyUtilib Component Architecture.

Enjoy!

Welcome to PyUtilib

These blogs contain information about PyUtilib releases, significant code developments and documentation updates. See the old PyUtilib blog for previous updates that were managed wiki pages.

  • Posted: 2010-03-28 13:19
  • Author: wehart
  • Categories: (none)
  • Comments (0)