Talk:Spec:Market

= Comments =

Market clearing
In the figure of market clearing, there should be vertical segments on the supply and demand curves. The supply vertical segment on the top end goes to $$+\infty$$ and the demand vertical segment on the bottom end goes to $$-\infty;$$. --Dchassin 15:51, 6 May 2010 (UTC)
 * This was effectively addressed by adding some extra verbage to the beginning of the market clearing section. --Ftuffner 19:39, 7 May 2010 (UTC)

Statistics interval
The statistics intervals should not be indexed explicitly. Instead the intervals should be assigned to a string, e.g., "1h 1d 3d 1w", and the appropriate variables published, e.g., "mean_1h", "stdev_1h", etc. --Dchassin 15:51, 6 May 2010 (UTC)
 * 1) The idea of dynamic publishing of variables by the object itself won't fly very far.  The interval string would have to be parsed in init, a long time after when the object is instantiated. Mhauer 23:54, 6 May 2010 (UTC)
 * 2) * Indeed, the size of the object must be known prior to the module loader caller the constructor. This isn't a problem if the interval string is a module global variable set before the constructor is called.  We should check that this is the case (I think it is, but I'm not certain about that).  If so, I suggest that the interval string be "1d" by default, but that we allow multiple variable that are space or comma delimited. If not, we need to find another way to make this user controlled.  The statistics have very large impact on the behavior of the controllers and should not be hard-coded.  I still think that explicit indexing is inflexible and weak.  Hopefully it will not be necessary. --Dchassin 15:47, 7 May 2010 (UTC)
 * 3) Possibly part of what Mhauer has indicated above, the dynamic output will cause unnecessary difficulties in the end-use devices of the market.  For example, an appliance controller would have to parse and fully decode the output string before performing its decision.  Aggregate this among a population, and it might start eating a significant chunk of simulation.  If published as a direct numerical value, they can easily be read by other objects. --Ftuffner 01:02, 7 May 2010 (UTC)
 * 4) * As far as I can see there is no additional processing burden from this above what is currently spec'd--the mean and variance must be computed for a series of samples collected over time. In fact wise implementation would run a single pass and simply fill the target mean/stdev as it goes down the list sorted by interval from shortest to longest. --Dchassin 15:47, 7 May 2010 (UTC)
 * 5) * This was not meant as a processing burden for the statistics. I agree, this shouldn't be any different really from the existing specification.  My point is controller objects or other devices elsewhere in the .GLM file that use these statistics.  Rather than just reading a simple published number field, they would need to parse out this text string to find the statistic they are interested in. --Ftuffner 19:39, 7 May 2010 (UTC)
 * 6) * I don't think so. Suppose I created a GLM file that needed to interval statistic, the last hour and the last day.  I would load the module with  .  Then the controller that needed the hourly mean would be defined to access mean_price_1h and the controller that needed the daily standard deviation would access stdev_price_1d.  It's a lot clearer than accessing mean_price_1 and stdev_price_2.  There's no need to parse the text string to find the statistic.  The user defines what is available from the market and the user defines what is needed by the controller.  Even if they're not the same user, that shouldn't be very troublesome to work out. --Dchassin 19:48, 7 May 2010 (UTC)
 * 7) * This was a misunderstanding of how you were publishing them. I was under the impression they were going to be published as an output string, similar to how they were input (e.g.,  .  As directly published variables, I agree it should not be a problem. --Ftuffner 19:53, 7 May 2010 (UTC)
 * 8) Another item with this dynamic string idea: Is there going to be a cap on the number of intervals?  The output will undoubtedly be a little larger than the input, so this probably needs to be capped at a reasonably low number. --Ftuffner 01:02, 7 May 2010 (UTC)
 * 9) * I don't think a sample cap is strictly necessary because it is easy to determine in advance what a particular interval string requires and dynamically allocate it. It's not like there are going to be a particularly large number of distinct market object compared to the number of controllers. But it is probably a good idea.  Recommend we cap it at 8192 samples (which is more than a month at 5 minutes). --Dchassin 15:47, 7 May 2010 (UTC)
 * 10) Some in-person conversation netted the idea of appending the desired mean and stdev properties into the relevant GLM file with a   chunk.  If the auction object can be configured to scan its own property list and identify the stdev and mean aggreator variables to fill in a dynamic fashion, the above requirement becomes much less complicated. Mhauer 22:27, 11 May 2010 (UTC)
 * 11) * Indeed. This means that statistic_interval_x is no longer required and that price_mean_x and price_stdev_x are user defined rather than formally defined.  The convention for x should be specified.  Suggest using the same convention as the tape player time step, e.g., 1d for 1 day, 2h for 2 hours, etc. --Dchassin 15:30, 12 May 2010 (UTC)
 * 12) * Can anyone think of a reason to collect mean and stdev statistics on quantity? --Dchassin 15:30, 12 May 2010 (UTC)
 * 13) * I can see statistics on the quantity being useful on the controller and market users' side of things (i.e., tracking their own quantities for refining their future bids). Market-level implementations may be something a more advanced controller could use in the future, but I would contend that the controller itself can track this quantity statistic if necessary (or it can be added at a future date). --Ftuffner 16:55, 13 May 2010 (UTC)
 * 14) * I can see it being an interesting feature, and it would allow some analysis of economic statistics, but I don't have any ideas on what it would be used for in the interim. Mhauer 19:37, 13 May 2010 (UTC)
 * 15) Should the auction module wait until the desired number of market clearings have past before calculating the mean and stdev? Basically a market initialization buffer zone. --andyfisher 23:11, 12 May 2010 (UTC)
 * 16) * I see this as one option. Another option is populating the circular buffer (assuming it is implemented this way) with the first cleared values, or known "invalid data" values.  Depending on the choice, this could either heavily bias the first x-values of the mean and standard deviation, or have the values not be representative of the past y-time periods.  There may even be a combination of andyfisher's idea and these (with some form of "valid calculation" flag set when the mean and standard deviation calculations are sufficiently populated). --Ftuffner 16:49, 13 May 2010 (UTC)
 * 17) * The user will have to set the initial values of the mean and the stdev for the controllers to work properly during the initialization period. --Dchassin 20:35, 13 May 2010 (UTC)
 * 18) * To be clear, will these user specified values will be held until the market is fully populated? --andyfisher 20:59, 13 May 2010 (UTC)
 * 19) * Valid question. Assuming a circular buffer for the measured values, should the full buffer be populated with the initial value and used as if valid from the start of the simulation, or should the initial value be force-fed to any inquiring values during the interim warm-up period?  Is this warm-up period defined in measurement cycle, in seconds, until there are no invalid data values, or by some other method? -Mhauer 19:30, 17 May 2010 (UTC)
 * 20) * The thought on this was to have one buffer with everything pre-populated at the fixed value. The statistic calculation would then be immediately started (so all standard deviations would start at zero).  This will allow a much more controlled, gradual transition into the "true" statistics, rather than a sudden, discrete jump when the statistic interval is satisfied. -Ftuffner 00:56, 18 May 2010 (UTC)

Max_quantity
Would buyer_max_quantity and seller_max_quantity make more sense as buyer_total_quantity and seller_total_quantity? 'max' implies that it is the maximum of a set, rather than the sum of a set of values. -Mhauer 23:28, 17 May 2010 (UTC)
 * 1) This may work. My issue with it is it is the complete sum of all of the quantities, not just those that cleared.  If buyer_total_quantity makes sense there, we can certainly change the spec. -Ftuffner 00:53, 18 May 2010 (UTC)
 * 2) Although with "clearing quantity", it should be implied that neither buyer nor seller quantity is the clearing quantity, since it would make little sense to have an additional field that could ambiguously duplicate one of two other fields. -Mhauer 18:52, 18 May 2010 (UTC)
 * 3) This was changed, as per discussions, to seller_total_quantity and buyer_total_quantity. -Ftuffner 23:03, 21 May 2010 (UTC)

Dirty statistics intervals
Consider the situation where a market clears every 15 minutes, and a 200 minute average is desired. Should the sample from 195 minutes prior to a when a given statistic is measured used that entire sample, effectively extending the statistic duration to 210 minutes, or should it calculate the average with only five minutes of the oldest sample applied? -Mhauer 21:04, 19 May 2010 (UTC)
 * Offline discussions indicated that statistics should only be calculable on integer intervals. This integer value will be calculated as the floor of the number of markets the statistic overlaps.  This integer must be at least one.  For example, suppose a market has 30 minute periods and 15 minute mean interval.  This should return an error because the statistic interval must be at least one market period (30 minutes).  Suppose in a similar example the market has 30 minute periods and a 75 minute mean is desired.  In this case, the statistic spans 2.5 markets.  However, the interval will be rounded down to 2 markets (60 minutes) and the statistic calculated that way.  This rounding operation should be presented to the user as a warning. -Ftuffner 21:51, 21 May 2010 (UTC)

Next market statistics with zero latency
In the very possible event that there is no latency on the market, and therefore no forward determination of the market behavior (or its statistics), what should be done with any 'next' statistics? Given a latency of 0, any market that clears should have its values fed directly into the current statistics, and the old statistics fed into the previous statistics. There would be no values for any forward-looking statistic fields. -Mhauer 21:24, 19 May 2010 (UTC)
 * Next market statistics were removed from the specifications due to complexities in their implementation, as well as no justification for their inclusion. -Ftuffner 23:03, 21 May 2010 (UTC)

Missing values
market_id, a core part of the original auction behavior, is not spec'ed in. network has vanished. linkref has been discussed offline. The demand and supply subtotals have vanished, while the totals have been renamed. market_id should be spec'ed in, and something about network and linkref (and how unresponsive bids are determined and added) should be mentioned either on the talk page or in the spec.
 * linkref has been generalized into capacity_reference_object and capacity_reference_property. -Ftuffner 23:03, 21 May 2010 (UTC)
 * market_id has been rolled into a bid id. It is discussed in the first paragraph of the Bid Acceptance section.  The implementation has been discussed briefly offline.  The thought was to return a unique bid id that is unique to a market.  The bid id would basically be the bid number in the current market, offset by an appropriate value to distinguish it from the previous market.  This could be the total number of bids in the previous market.  e.g., 100 objects bid into the previous market.  The third object to bid into the current market receives a unique identifier of 103 (100 previous offset + 3rd bid).  The actual implementation can vary a bit, but this was the overall idea. -Ftuffner 23:03, 21 May 2010 (UTC)

Latent market warm-up period
What output values are appropriate during the initial latency period of the market, where the cleared markets have not yet gone active? -Mhauer 20:29, 21 May 2010 (UTC)
 * Initial latency will be handled like "long" statistics. The initial value of init_price will initialize the statistic array.  It will remain this value until the latency expires and the markets become active.  At this point, the "calculation array" will slowly be replaced by valid data points and update the statistic appropriately. -Ftuffner 23:03, 21 May 2010 (UTC)

capacity_reference_property type constraints?
Should the capacity_reference_property be restricted to PT_double and PT_complex for the sake of sanity?
 * This seems like a reasonable restriction for GridLAB-D at this time.-Ftuffner 01:05, 10 June 2010 (UTC)

'None' clearing type price determination scalar
In the spirit of avoiding magic numbers, how about a scalar property that determines the weight of the calculation of the clearing price when the market does not clear? The default would be 0.5, while increasing the value steers it towards the sell price, and decreasing values dive towards the buy price. This value would be bound to (0.0, 1.0).
 * I assume this is actually for the market NULL clearing type. If I follow correctly, this would be a way to adjust the clearing price from the mid-point between the two curves from midpoint.  I'm not sure I follow why this may be needed. -Ftuffner 01:05, 10 June 2010 (UTC)

init_price
There have been questions for if init_price should be used for the clearing price, along with the average price, when the market has not cleared any frame, due to just starting up or due to market latency.

Single-sided market documentation
Some additional clearing situations were identified with the fixed price and fixed quantity markets. In the case where the fixed price market does not find any bids, it will clear at that fixed price with a zero quantity. In the case where a fixed quantity market does not clear its entire quantity, whether buying or selling, the market fails to clear ~ "what is the intersection of two parallel lines?" -DC. The single auction markets will fall under existing clearing cases should there be adequate bids for the fixed quantity, or cleared bids for the fixed price.Mhauer 00:20, 3 July 2010 (UTC)

There is a distinct lack of documentation on single-sided market behavior, including that the value of fixed_price at the clearing time is used, rather than at the time the bids were taken. This was causing some confusion when a test script was being written by someone who has been writing double auction scripts, and has gotten used to having the price preceding the clearing time be the clearing price, rather than the price at the clearing time. -Mhauer 21:10, 9 August 2010 (UTC)