Printed from https://fyi.org.nz/request/8425-a-file-of-half-hourly-data-to-be-published-monthly-is-omitted?unfold=1 on April 19, 2024 00:20

A file of half-hourly data to be published monthly is omitted

Nicky McLean made this Official Information request to Electricity Authority

The request was partially successful.

From: Nicky McLean

Dear Electricity Authority,
Via your web site https://www.emi.ea.govt.nz/Wholesale/Dat...
there should appear a file every month supplying the previous month's half-hourly data, as with file 201806_Final_prices.csv via https://www.emi.ea.govt.nz/Wholesale/Dat...
But file 201805_Reserve_prices.csv is the last one present and file 201806_Reserve_prices.csv is absent.

1) Can this file be supplied?
2) Or, has there been a shift in policy to not supply it?
3) By what date in the month ought these monthly data files be expected to be supplied?
3) What procedures are being followed to ensure that the standard monthly supply of data files is being performed correctly?
4) Is this event unique? What correspondence on data quality, coverage and supply have you received?

Yours faithfully,
R.N. McLean

Link to this

From: Info Electricity Authority
Electricity Authority

Dear Nicky

Thank you for your below email, which I am forwarding to the Market Information team to follow up on. They will be in touch as soon as practicable.

Best wishes for a good weekend.

 Kind regards

    Suzanne Heyworth
         Organisation Support Administrator                 
         DDI:      +64 4 471 8626                                                  
         Email:   [email address]    

         Electricity Authority - Te Mana Hiko                                    
         Level 7, ASB Bank Tower, 2 Hunter Street
         PO Box 10041
         Wellington 6143
         New Zealand
         www.ea.govt.nz
       
            

-----Original Message-----
From: Nicky McLean [mailto:[FOI #8425 email]]
Sent: Thursday, 2 August 2018 10:05 PM
To: Info Electricity Authority
Subject: Official Information request - A file of half-hourly data to be published monthly is omitted

Dear Electricity Authority,
Via your web site https://www.emi.ea.govt.nz/Wholesale/Dat...
there should appear a file every month supplying the previous month's half-hourly data, as with file 201806_Final_prices.csv via https://www.emi.ea.govt.nz/Wholesale/Dat...
But file 201805_Reserve_prices.csv is the last one present and file 201806_Reserve_prices.csv is absent.

1) Can this file be supplied?
2) Or, has there been a shift in policy to not supply it?
3) By what date in the month ought these monthly data files be expected to be supplied?
3) What procedures are being followed to ensure that the standard monthly supply of data files is being performed correctly?
4) Is this event unique? What correspondence on data quality, coverage and supply have you received?

Yours faithfully,
R.N. McLean

-------------------------------------------------------------------

This is an Official Information request made via the FYI website.

Please use this email address for all replies to this request:
[FOI #8425 email]

Is [Electricity Authority request email] the wrong address for Official Information requests to Electricity Authority? If so, please contact us using this form:
https://fyi.org.nz/change_request/new?bo...

Disclaimer: This message and any reply that you make will be published on the internet. Our privacy and copyright policies:
https://fyi.org.nz/help/officers

If you find this service useful as an Official Information officer, please ask your web manager to link to us from your organisation's OIA or LGOIMA page.

-------------------------------------------------------------------

------------------------------------------------------------------
"The information contained in this transmission is confidential.
It is intended for the named addressee only.
If you are not the named addressee you may not copy, distribute or
take any action in reliance upon this transmission."
------------------------------------------------------------------

hide quoted sections

Link to this

From: Info Electricity Authority
Electricity Authority

Hi Nicky

On behalf of our Market Information team, I am happy to advise you that the information you were seeking is in the following link -

https://www.emi.ea.govt.nz/Wholesale/Dat...

Hope this is helpful.

 Kind regards

    Suzanne Heyworth
         Organisation Support Administrator                 
         DDI:      +64 4 471 8626                                                  
         Email:   [email address]    

         Electricity Authority - Te Mana Hiko                                    
         Level 7, ASB Bank Tower, 2 Hunter Street
         PO Box 10041
         Wellington 6143
         New Zealand
         www.ea.govt.nz
       
            

-----Original Message-----
From: Info Electricity Authority
Sent: Friday, 3 August 2018 5:31 PM
To: Nicky McLean
Cc: emi; Info Electricity Authority
Subject: RE: Official Information request - A file of half-hourly data to be published monthly is omitted
Importance: High

Dear Nicky

Thank you for your below email, which I am forwarding to the Market Information team to follow up on. They will be in touch as soon as practicable.

Best wishes for a good weekend.

 Kind regards

    Suzanne Heyworth
         Organisation Support Administrator                 
         DDI:      +64 4 471 8626                                                  
         Email:   [email address]    

         Electricity Authority - Te Mana Hiko                                    
         Level 7, ASB Bank Tower, 2 Hunter Street
         PO Box 10041
         Wellington 6143
         New Zealand
         www.ea.govt.nz
       
            

-----Original Message-----
From: Nicky McLean [mailto:[FOI #8425 email]]
Sent: Thursday, 2 August 2018 10:05 PM
To: Info Electricity Authority
Subject: Official Information request - A file of half-hourly data to be published monthly is omitted

Dear Electricity Authority,
Via your web site https://www.emi.ea.govt.nz/Wholesale/Dat...
there should appear a file every month supplying the previous month's half-hourly data, as with file 201806_Final_prices.csv via https://www.emi.ea.govt.nz/Wholesale/Dat...
But file 201805_Reserve_prices.csv is the last one present and file 201806_Reserve_prices.csv is absent.

1) Can this file be supplied?
2) Or, has there been a shift in policy to not supply it?
3) By what date in the month ought these monthly data files be expected to be supplied?
3) What procedures are being followed to ensure that the standard monthly supply of data files is being performed correctly?
4) Is this event unique? What correspondence on data quality, coverage and supply have you received?

Yours faithfully,
R.N. McLean

-------------------------------------------------------------------

This is an Official Information request made via the FYI website.

Please use this email address for all replies to this request:
[FOI #8425 email]

Is [Electricity Authority request email] the wrong address for Official Information requests to Electricity Authority? If so, please contact us using this form:
https://fyi.org.nz/change_request/new?bo...

Disclaimer: This message and any reply that you make will be published on the internet. Our privacy and copyright policies:
https://fyi.org.nz/help/officers

If you find this service useful as an Official Information officer, please ask your web manager to link to us from your organisation's OIA or LGOIMA page.

-------------------------------------------------------------------

------------------------------------------------------------------
"The information contained in this transmission is confidential.
It is intended for the named addressee only.
If you are not the named addressee you may not copy, distribute or
take any action in reliance upon this transmission."
------------------------------------------------------------------

hide quoted sections

Link to this

From: Nicky McLean

Dear Ms. Heyworth,
Thank you for your reply of Tuesday the 9'th; the missing data file 201806_Reserve_prices.csv had appeared on Monday the 7'th with a timestamp of around noon - so it has taken two days to not respond to any other part of the request. Ought I also pretend not to notice?
The data file appears valid, though starting it with a Byte Order Mark (hexadecimal EF BB BF) is quite unnecessary for a plain ASCII text file that does not involve Unicode. All the other data files supplied via the web page do not have this BOM.

Yours sincerely,

Nicky McLean

Link to this

From: Nicky McLean

Dear Info Electricity Authority,
Oops, please read Thursday the ninth of August, instead of "Tuesday"...

Yours sincerely,

Nicky McLean

Link to this

From: Andrew Springett
Electricity Authority

Attachment image001.jpg
0K Download

Good afternoon Nicky

 

I refer to your emails of 2 and 12 August 2018 relating to half hour data
that is published on the Authority's website, on a monthly basis. I have
been asked to assist with responding to your latest request as it appears
that it may relate to a request under the Official Information Act.

 

In your original email you asked 5 questions. I respond to each in turn:

 

1) Can this file be supplied?

We have since provided the link to the file you requested.

 

2) Or, has there been a shift in policy to not supply it?

The file you requested is available on our website.  There has been no
shift in policy.

 

3) By what date in the month ought these monthly data files be expected to
be supplied?

The Authority receives these files from an external service provider. We
put them onto our website within 2 working days of receiving the file from
the external provider.

 

4) What procedures are being followed to ensure that the standard monthly
supply of data files is being performed correctly?

There are no formal procedures in place. As noted above, we receive the
data files from an external provider and we then place them on our
website.

 

5) Is this event unique? What correspondence on data quality, coverage and
supply have you received?

We are not sure what event you refer to here. If you are able to clarify
this part of your email, we will be able to consider providing the
information you are looking for.

 

Regards

Andrew

 

[1]cid:image001.jpg@01CF4744.F29D3150     Andrew Springett      

         Senior Legal Counsel                 

 

-----Original Message-----

From: Nicky McLean
[[2]mailto:[FOI #8425 email]]

Sent: Sunday, 12 August 2018 11:17 PM

To: Info Electricity Authority

Subject: RE: Official Information request - A file of half-hourly data to
be published monthly is omitted

 

Dear Ms. Heyworth,

   Thank you for your reply of Tuesday the 9'th; the missing data file
201806_Reserve_prices.csv had appeared on Monday the 7'th with a 
timestamp of around noon - so it has taken two days to not respond to any
other part of the request. Ought I also pretend not to notice?

   The data file appears valid, though starting it with a Byte Order Mark
(hexadecimal  EF BB BF) is quite unnecessary for a plain ASCII text file
that does not involve Unicode. All the other data files supplied via the
web page do not have this BOM.

 

Yours sincerely,

 

Nicky McLean

 

-----Original Message-----

 

Hi Nicky

 

On behalf of our Market Information team, I am happy to advise you that
the information you were seeking is in the following link -

 

[3]https://www.emi.ea.govt.nz/Wholesale/Dat...

 

 

Hope this is helpful.

 

 Kind regards

 

    Suzanne Heyworth

         Organisation Support Administrator                 

         DDI:      +64 4 471 8626                  
                               

         Email:   [email address]    

 

         Electricity Authority - Te Mana Hiko   
                                

         Level 7, ASB Bank Tower, 2 Hunter Street

         PO Box 10041

         Wellington 6143

         New Zealand

         [4]www.ea.govt.nz

       

            

 

-------------------------------------------------------------------

Please use this email address for all replies to this request:

[5][FOI #8425 email]

 

Disclaimer: This message and any reply that you make will be published on
the internet. Our privacy and copyright policies:

[6]https://fyi.org.nz/help/officers

 

If you find this service useful as an Official Information officer, please
ask your web manager to link to us from your organisation's OIA or LGOIMA
page.

 

-------------------------------------------------------------------

 

------------------------------------------------------------------
"The information contained in this transmission is confidential.
It is intended for the named addressee only.
If you are not the named addressee you may not copy, distribute or
take any action in reliance upon this transmission."
------------------------------------------------------------------

References

Visible links
2. mailto:[FOI #8425 email]
3. https://www.emi.ea.govt.nz/Wholesale/Dat...
4. http://www.ea.govt.nz/
5. mailto:[FOI #8425 email]
6. https://fyi.org.nz/help/officers

hide quoted sections

Link to this

From: Nicky McLean

Dear Andrew Springett,
Thank you for your response; as acknowledged previously, the missing file has been supplied on your organisation's website and so the main issue, its absence, is resolved by its manifestation. This also indicates that there has been no policy change to not supply it.

Concerning 3), recursive logic suggests that resolving the query becomes a matter of when the external data supplier is expected to supply its data files to your organisation. If a steady two-day processing time is allowed, this suggests a rather variable delivery date since your organisation has published data files as early as the 12'th or 13'th of the month as well as towards the end of a month, or later...

Concerning 5), "this event" refers to the non-supply of file 201806_Reserve_prices.csv along with the other files that were supplied in July containing data for June, which were datestamped Tuesday 24'th July. While this event in July is unique in itself, the implication I intended was to ask whether there have been other occasions when the monthly supply of data files has omitted one or more such files. To avoid confusion, it refers not just to the files of the Reserve_prices collection alone, but others too, and I should have made that part more clear. There are nine such files:
~_Grid_export.csv,
~_Grid_export_to_Tiwai.csv,
~_Grid_import.csv,
~_HVDC_flows.csv,
~_Reactive_power.csv,
~_Embedded_generation.csv,
~_Unit_level_generation.csv,
~_Reserve_prices.csv,
~_Final_prices.csv
where the ~ represents an appropriate date such as 201806, and which is adjusted for the appropriate year and month.

So then, has any other communication been received concerning this event, and, if the event is not unique (extending now to all nine data streams and for other months), have there been any other communications about omitted data? Concerning data supply, coverage, and quality?
Like, does anyone notice? Your response to 4) and 3) suggests that your own staff do not, merely transferring whatever is supplied, or not, to your web page.

Yours sincerely,

Nicky McLean

Link to this

From: Andrew Springett
Electricity Authority

Hi Nicky
Thanks for your email below clarifying your original request.
In response to clarified question 5 from your original request, we have no record of having received any other communications about this data having been omitted, or concerns about the data's supply, coverage, or quality.
Regards,
Andrew

-----Original Message-----
From: Nicky McLean [mailto:[FOI #8425 email]]
Sent: Tuesday, 28 August 2018 7:10 PM
To: Andrew Springett
Subject: Re: FW: Official Information request - A file of half-hourly data to be published monthly is omitted

Dear Andrew Springett,
Thank you for your response; as acknowledged previously, the missing file has been supplied on your organisation's website and so the main issue, its absence, is resolved by its manifestation. This also indicates that there has been no policy change to not supply it.

Concerning 3), recursive logic suggests that resolving the query becomes a matter of when the external data supplier is expected to supply its data files to your organisation. If a steady two-day processing time is allowed, this suggests a rather variable delivery date since your organisation has published data files as early as the 12'th or 13'th of the month as well as towards the end of a month, or later...

Concerning 5), "this event" refers to the non-supply of file 201806_Reserve_prices.csv along with the other files that were supplied in July containing data for June, which were datestamped Tuesday 24'th July. While this event in July is unique in itself, the implication I intended was to ask whether there have been other occasions when the monthly supply of data files has omitted one or more such files. To avoid confusion, it refers not just to the files of the Reserve_prices collection alone, but others too, and I should have made that part more clear. There are nine such files:
~_Grid_export.csv,
~_Grid_export_to_Tiwai.csv,
~_Grid_import.csv,
~_HVDC_flows.csv,
~_Reactive_power.csv,
~_Embedded_generation.csv,
~_Unit_level_generation.csv,
~_Reserve_prices.csv,
~_Final_prices.csv
where the ~ represents an appropriate date such as 201806, and which is adjusted for the appropriate year and month.

So then, has any other communication been received concerning this event, and, if the event is not unique (extending now to all nine data streams and for other months), have there been any other communications about omitted data? Concerning data supply, coverage, and quality?
Like, does anyone notice? Your response to 4) and 3) suggests that your own staff do not, merely transferring whatever is supplied, or not, to your web page.

Yours sincerely,

Nicky McLean

-----Original Message-----

Good afternoon Nicky

 

I refer to your emails of 2 and 12 August 2018 relating to half hour data
that is published on the Authority's website, on a monthly basis. I have
been asked to assist with responding to your latest request as it appears
that it may relate to a request under the Official Information Act.

 

In your original email you asked 5 questions. I respond to each in turn:

 

1) Can this file be supplied?

We have since provided the link to the file you requested.

 

2) Or, has there been a shift in policy to not supply it?

The file you requested is available on our website.  There has been no
shift in policy.

 

3) By what date in the month ought these monthly data files be expected to
be supplied?

The Authority receives these files from an external service provider. We
put them onto our website within 2 working days of receiving the file from
the external provider.

 

4) What procedures are being followed to ensure that the standard monthly
supply of data files is being performed correctly?

There are no formal procedures in place. As noted above, we receive the
data files from an external provider and we then place them on our
website.

 

5) Is this event unique? What correspondence on data quality, coverage and
supply have you received?

We are not sure what event you refer to here. If you are able to clarify
this part of your email, we will be able to consider providing the
information you are looking for.

 

Regards

Andrew

 

[1]cid:image001.jpg@01CF4744.F29D3150     Andrew Springett      

         Senior Legal Counsel                 

 

-------------------------------------------------------------------
Please use this email address for all replies to this request:
[FOI #8425 email]

Disclaimer: This message and any reply that you make will be published on the internet. Our privacy and copyright policies:
https://fyi.org.nz/help/officers

If you find this service useful as an Official Information officer, please ask your web manager to link to us from your organisation's OIA or LGOIMA page.

-------------------------------------------------------------------

------------------------------------------------------------------
"The information contained in this transmission is confidential.
It is intended for the named addressee only.
If you are not the named addressee you may not copy, distribute or
take any action in reliance upon this transmission."
------------------------------------------------------------------

hide quoted sections

Link to this

From: Nicky McLean

Dear Andrew Springett,
Thank you for your reply of the thirtieth of August, which has confirmed what I had begun to suspect from the lack of response (not even any acknowledgements) to queries made via other means, which is that they were immediately filed in the Round Receptacle, ignored, then forgotten.

Yours sincerely,
Nicky McLean

Link to this

Things to do with this request

Anyone:
Electricity Authority only: