Ticket #4493 (new defect)

Opened 5 years ago

Last modified 4 years ago

Incorrect / odd behavior when manipulating indexed sets

Reported by: jwatson Owned by: unassigned
Priority: major Milestone: Long Term
Component: pyomo.core Version:
Keywords: Cc:


Consider the following fragment:

model = ConcreteModel?()
model.a_set = Set([1,2])

This yields the error:

KeyError?: 'Cannot access index 1 in array set a_set'

I would argue that the sets indexed by 1 and 2 should be defined, and should be by default empty.

Change History

comment:1 Changed 5 years ago by wehart

This semantics is consistent with the rest of Pyomo: A Set declaration specifies the range of valid sets and not the range of actual sets. Thus,

model = ConcreteModel()
model.A = Set([1,2])

should create an empty array of sets, for which the indices 1 and 2 are valid during construction.

HOWEVER, if a user does


then we should interpret that to mean that they want to initialize the 1-th index of the Set array. Can we do that automatically? I'm not sure. The A[] index retrieves a value, so the KeyError? makes sense. We could have a syntax like this, though

model.A.initialize(1, set(5.0))

to initialize a specific set for a given index.

comment:2 Changed 5 years ago by wehart

  • Milestone set to Coopr 3.5

comment:3 Changed 5 years ago by wehart

  • Milestone changed from Coopr 3.5 to Coopr 3.6

comment:4 Changed 4 years ago by wehart

  • Milestone Coopr 3.6 deleted

Milestone Coopr 3.6 deleted

comment:5 Changed 4 years ago by wehart

  • Milestone set to Pyomo 4.x

comment:6 Changed 4 years ago by jdsiiro

I am not sure I follow Bill's argument. It looks like that constructing a Set without an initialize keyword does not actually construct the set. If I take the original example and add the initialize argument, then things work just fine:

>>> model = ConcreteModel()
>>> model.a_set = Set([1,2], initialize=[])
>>> model.a_set[1].add(5.0)

This behavior doesn't seem to match what we do with Var:

>>> model = ConcreteModel()
>>> model.x = Var([1,2])
>>> print model.x[1]
>>> print value(model.x[1])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../pyomo/src/pyomo/pyomo/core/base/numvalue.py", line 159, in value
    % (obj.cname(),))
ValueError: No value for uninitialized NumericValue object x[1]
>>> model.x[1] = 5
>>> print value(model.x[1])

The ValueError makes sense, as the (constructed) variable has no value -- but it is not the KeyError we get from Set:

>>> model = ConcreteModel()
>>> model.a_set = Set([1,2])
>>> print model.a_set[1]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../pyomo/src/pyomo/pyomo/core/base/indexed_component.py", line 240, in __getitem__
    return self._default(ndx)
  File ".../pyomo/src/pyomo/pyomo/core/base/sets.py", line 1612, in _default
    "Index %s not defined for set array %s" % (index, self.__class__.__name__,))
KeyError: 'Index 1 not defined for set array IndexedSet'

I would think that the correct behavior is for constructing an indexed set with no initialize argument should initialize the set to the empty set. I think this is a pretty trivial fix:

  • pyomo/core/base/sets.py

    16091609        This returns an exception. 
    16101610        """ 
    1611         raise KeyError( 
    1612             "Index %s not defined for set array %s" % (index, self.__class__.__ 
     1611        tmp = self._data[index] = self._SetData(self, self._bounds) 
     1612        return tmp 
    16141614    def __setitem__(self, key, vals): 
    16151615        """ 

comment:7 Changed 4 years ago by jdsiiro

  • Priority changed from normal to major
  • Milestone changed from Pyomo 4.x to Pyomo 4.1

comment:8 Changed 4 years ago by jdsiiro

To complete a consistency argument: I think a sparse indexed Set should behave similarly to a sparse indexed (mutable) Param with a default value: that is, if the data only provided subset information of some of the indices, then only those indices should be created. Iterating over the indexed set will only yield the indices that were specified in the data.

However, if a user requests other indices later, then we should create the index value and initialize it to an empty set. This is consistent with the user's definition: the indexed set should be valid over all indices used to declare it (although some of those indices should map to empty subsets). If the user really didn't want those indices to be valid, then they should have constructed/filtered the indexing set appropriately.

comment:9 Changed 4 years ago by jdsiiro

Referenced in changeset [10347]:

An indexed Set that was only partially constructed (that is, not all of the valid indices had subsets associated with them) should implicitly add empty subsets to the unconstructed indices when accessed by the user. This resolves #4493.

comment:10 Changed 4 years ago by gahacke

I don't think the issue is fully addressed (see #4560). That ticket highlights a strange side-effect of loading data through a DAT file that causes setdata indices to be deleted when they are not declared in the DAT file (even if the initialize keyword is used in the model). The end result is a KeyError? just as unintuitive as the one for this ticket.

comment:11 Changed 4 years ago by wehart

  • Milestone changed from Pyomo 4.1 to Long Term
Note: See TracTickets for help on using tickets.