Json API to pull usage data

[IT_Guy]

Hi Agl

Is there an API available so i can pull my usage data ( same data found in the downloadable scv at https://myaccount.agl.com.au/usage )  from my agl account to a database i'm creating to analyse my solar generation across multiple system and my power usage 

[CozmoNz]

An alternative with API Access is Amber Electric. Its wholesale pricing though....

[dansten1]

@David_AGL Hey David, its been 3 years now, what is the progress of this?

[David_AGL]

Hi @dansten1 , thanks for checking in. We're certainly aware of the popularity of this request and there's plenty of people on the AGL team who would be excited to implement it, but finding the resources required to develop it among competing priorities has been a challenge. I'm always keen to present more of the customer context when I share this idea though: What would your main uses for API access be? Do you use any existing solutions for home automation or tracking your usage data, outside of the AGL app and My Account?

[thisrandomdude7]

I think a lot people would love to integrate this with their Home Assistant instance. It would be very useful to at least:

- Be able to pull the account tariff, pricing, hours

- Pull daily/hourly energy usage for electricity and gas (if available) to calculate live-ish power consumption, without all the historical usage going more than 6 months back

The CSV ZIP download requires Bearer auth and isn't straightforward on how to integrate it. It's not automation friendly.

Imo best idea would be to look at what HASS offers and start there, i.e. gas, solar and electricity usage/prices/tariffs/return

https://www.home-assistant.io/docs/energy/

https://www.home-assistant.io/blog/2021/08/04/home-energy-management/

Some other relevant links on other implementations of similar features in other countries:

https://www.home-assistant.io/integrations/pvpc_hourly_pricing/ Spain electricity hourly pricing (PVPC)

https://www.home-assistant.io/integrations/amberelectric/ Amber Electric implementation of the component / their API

https://www.home-assistant.io/integrations/#search/energy components for API access of multiple other energy companies

Hope this helps

[David_AGL]

Thanks @thisrandomdude7 , that's excellent

[NeilC]

Hi all,

You will all have trouble getting an API in place as AGL does not seem to have the technical expertise to complete many tasks and seems to carry out tasks without applying the necessary change controls and processes.

I have pointed out to them that when Daylight Savings starts in October the clock goes forward one hour and the CSV file misses one hour.

Above is the data from my usage file for the 2nd October 2022

If you have looked at the file you will know that the time shown is 01:59:59 (really 02:00)

Now I used .089kWh for the half hour from 01:30 to 02:00 and .085kWh for the half hour from 03:00 to 03:30 nothing for the missing hour and hence only 23 hours in that day.

Above you can see that my usage roughly doubled when we lost the hour when DST went back an hour. Why did we not have a 25 hour day then

I have been processing the file for some FOUR YEARS now using python (works in Linux and MS Windows).

Two programs, first converts the .CSV file into a NEM format file (the industry standard) and the second reads the NEM format file and creates user defined reports.

AGL has done lots of things without informing any one of the changes these include changing the time format in the CSV file from 24 hour to 12 hour AM / PM. My solution was to make my program be able to read either 12H or 24H time formats.

Then I discovered the daylight saving issue because I assumed that there was 24hours in a day, then of course the times specified are in Eastern Time (I live in SA).

Now having a day that is 24 hours long means that once I find the start of the file I simply need to read the next 48 entries for each sub meter to get a full day.

That is an issue when there is one hour missing on the first Sunday Morning in Oct. Its also an issue if your meter stops reporting for a time, hence my program now reads the whole file and confirms that each meter reports every half hour for the length of the file (UP TO TWO YEARS WORTH).

Then of course Midnight (23:59:59) is only 23:29:59 in SA so I have to start my count of 48 from the second reading of the file and have to skip the last day in the file as it ceases at Midnight Eastern time.

Hence I have to wait until the CSV file report the data for the DAY AFTER my billing cycle finishes to get the last day.

Here is a run of the file that processes the CSV file into the NEM format file you can see that it processed DST skips six times.

Once I have the data I can then process that into reports.

Note that the Meter Read is only the kWh used in this billing period as my meter has been consistently reset by AGL / SA Power networks since it has been installed. In fact for ONE DAY it was set to read to FOUR decimal points during  one reset.

My meter is also set to read every 5 minutes which means you get to pay even more for demand charges if you are silly enough to take that OPTION.

This is the data processed from my generated NEM file into this CSV Report which is the actual data that AGL uses to create my Bill (normally accurate to two cents).

You can do this at any time during the billing cycle or if you prefer for the whole of the last two years data...

Total time taken

Time to log on to AGL,download the file, go to downloads and unzip the file about 1m30sec.

Time to extract the data 2.7 seconds

Time to create the report .16 seconds

Total time to be able to read the created reports less than two Minutes.

Then all I have to do is cut and past the data into my ElectUsed spreadsheet.

Data above is for my last billing period that finished midnight 6th July.

And I have all the data since the 28th April 2019.

This is then parsed to my BILLING sheet along with all my other data will allows us to manage all of our billing (which we are in credit for agencies Water, Electricity, Rates, Emergency Services, etc).

Hope you all enjoy your wait for the API and then have to go through it to find out if they have errors...

All I have to do now is wait the nearly week before I get my Bill that I had all the data for on the 8th July.

By the way why don't you all ask why don't we get the NEM format file rather the lengthy CSV file.

All of these files contain the same DATA

You can extract the data very simply out of the NEM file because in all reality its just a spread sheet.

By the way I am a friend of this neighbourhood and have been for four years.

If you click the like for me I will be even happier.

[Stu323]

Is there another energy supply company that provides an API?

[NeilC]

@Stu323 

Hi,

There may be some Retailers that supply an API.

I found a free one on the internet and supplied 900K of data which they converted.

Besides being wrong it was completely perfect.

1. I got a Json file.

2. I compared it to the actual data I had.

3. It was wrong.

Its like driving your car down the road the speed limit is 50, you travel at 50 and every "Bstard" overtakes you. You turn off and do 55 and you get a speeding ticket.

Now when I am travelling at 50k's in my car I am doing about 48k's, at 80 I am only doing 77-80k's,  90  doing 88-87k's, etc etc.

Base on my 50k's I can travel legally up to 52k's and not get booked.

I can also tell you that my speedo has an error of about 3.69% (which means I have to multiply my current speed by 96.31%)

So 50kM per hour = 50 * .961 or believe it or not 48kM per hour.

My last car (which I owned for 8 years) had a speedo error of .98, so at 50kM per hour I was travelling at 49kph.

Do you want me to go into fuel usage compared with what the ELECTRONIC gauge (litres per 100k ) said.

Yep I can tell you that for both vehicles.

I decode the data that AGL supplies and the usage I have is very close to their data. Now I can tell you that I have found out that the data they supply to me is based on SA time and the Data I get from SA Power Networks is in EST (with NO DAYLIGHT TIME adjustments).

Now the only issue I have with AGL is that I can only calculate my bill to within $0.02 cents (in my favour).

Without DATA you cannot prove that what you are getting is what you are being supplied.

Here is some of my data from 2013 with my old car.

Dist	Real Dist	L/100Km	Average	Car LpM	Error	RealFuel
350	358	First Fill	N/A
675	690	8.41	8.59	8.1	4%	8.41
297	304	8.63	8.66	8.2	5%	8.63
361	369	8.47	8.66	8.2	3%	8.47
681	696	8.48	8.66	8.2	3%	8.48
670	685	8.50	8.67	8.2	4%	8.50
682	697	8.29	8.63	7.7	7%	8.29

Doing this I have to fill my vehicle to a specific place in the filling neck (which is above the breather pipe) and then rock the car until accepts no more fuel.

Above was in the first 5 months of owning the vehicle.

Well I am 27 days into my new bill and here are my stats.

Peak kW	ShoulderkW	O/peak kW	Export kW	Gen kW
75.252	3.148	30.967	312.919	400.4

Using the data to hand I can do this projection

Yep I need a JSON file to prove my data as well. How are you going to prove yours.

Any way, as to your question, have you read all the replies and seen who has said they have found one..

Here is a document from AEMO in PLAIN ENGLISH to help you in getting your answer.

The link is https://www.google.com.au/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwjSy8OEq8CAAxU2xjgGHUMhC... 

This will download a 90 page simple English document for you (click on the link and it will down load the PDF to your device).

By the way this is for the NEM format 12 or 13 format...

Don't forget to click the like button...

[vk5ztv]

Yes, multiple, both in Australia and overseas.

AUS:
Amber Electric

NORTH AMERICA:

Atlantic City Electric
Baltimore Gas and Electric
Commonwealth Edison
Delaware Power
Evergy
PECO Electric Company
Pacific Gas and Electric
Potomac Electric Power Company
Puget Sound Energy

EUROPE:
Discovergy (Germany/EU)
EasyEnergy (Netherlands)
EnergyVanOns (Netherlands)

These are just a few that are available as Home Assistant integrations - there
will be numerous others. Some give only pricing data, some give pricing and
usage data (far more useful).

It's not a new thing for energy wholesalers and retailers to do, giving access
to this type of data via a secure API.



--
==================================================================================================================
Rodney Baker
rodney.baker@iinet.net.au
==================================================================================================================

[dansten1]

@David_AGL 
to answer your question,

I use the incredibly popular Home Assistant platform to monitor and automate all areas of my home.

There must be a API serving data to the AGL mobile app Usage page today since I can see my usage data in there…

Why not expose/leverage this?