Using the API to get raw second data

[tlveik]

I just received and installed my new TED Pro Home and am working on my program to read raw second data out of the ECC using the API.  For some reason the time stamps in the raw data aren't consecutive.  Here is an example.

I'll send a command like this; history/export.raw?T=0&D=0&M=1&C=10&I=0

And I get time stamps like this.
1494707836
1494707835
1494707834
1494707833
1494707832
1494707841
1494707832
1494707831
1494707830
1494707829

I know how to decode this, it just seems odd that they are not consecutive.  I guess I could just ignore the records that seem to be out of place.  Other than that, it is working fine.  Anyone have any insight into this?

Tom

[tlveik]

More info...
This glitch I'm seeing in the time stamps is causing the TED to loose data over time.  I just read out 3600 data points of one-second data.  What I got was 3600 records, but they only cover 49 minutes of time.


I miss my old TED 5000.  It worked much better than my new TED PRO.

Tom

[Support7]

Hi Tom, I'm not the developer so I can't say for certain but my best guess would be that 49 minutes of data would work out to be 2,940 data points of second data leaving 660 of your mentioned 3600 data points unaccounted for so you've got about 11 minutes of the measured hour that is suspect. Thinking about how PLC (communication between MTU and ECC) works and the fact that it is subject to interference on the power line on which it travels and the potential for packet collision it is a safe assumption that you will have occasional skipped info packets from the MTU to the ECC which would explain the nonconsecutive time stamps and missing 11 minute time frame. 11 minutes out of the hour may seem like a lot if it was consecutive but it is far more likely to be spread out over the course of the hour. For example you may get 10 minutes of valid readings and then skip 1 minute due to communication issues relating to interference or something along those lines.

You can check your communication levels by going to Footprints > Help > TED Pro Statistics Page. The MTU Rec. represents the number of info packets received from the MTU and the MTU Skp. is the number of skipped packets. It is normal for the MTU Skp # to count up occasionally as there are too many variables to account for to expect an interference free power line. Normally we look for about a 75%-25% MTU Rec to Skp ratio but this is just a simple guideline as it is subjective to when your looking at it.

It's unfortunate that you feel that way about the TED Pro compared to the TED5000, have you tried reading the raw data points of the TED5000 like you are with the TED Pro? I ask because one of the main hardware upgrades between the two systems is the TED Pro has a much more robust PLC modem to allow for better communication then the TED5000 was capable of so you would have likely had more missing data points assuming the local environment is the same.

TED Support
www.theenergydetective.com
 

[tlveik]

Thanks for the reply.

The data records are not just non-consecutive, they are also out of order.  Take a look at those time stamps I posted above again.  It will jump ahead several seconds, then jump back to where it was.  It looks like it looses about 2 seconds every time it does that.

Yes, I read raw data out of the TED5000 for years and rarely had any problem.  I'd still be using it if it didn't finally fail.  There would be occasional glitches but never consistent 11 minute losses like I see now.

My Skp to Rec ratio looks like 27% at the moment.
Here is a screen shot of my stats page.  I reset the ECC this morning hoping that might help.  I suppose the counts would be larger otherwise.

Tom

[pfletch101]

A skip ratio of 27% is pretty high. I would try to get that down - certainly to below 20%, ideally to below 10%.

[tlveik]

Okay, I'll rearrange things and try to lower that ratio to see if that helps.

Tom

[tlveik]

I rearranged everything today.  Moved the ECC and the MTU to a different phase.  Unplugged a few things that I thought might be suspect noise producers.

Initially it looked like there may be some improvement in the skip ratio but it returned to about 27% again.  Still loosing 11 minutes of data in each API 3600 second raw data read.

By the way, the same data loss shows up in the Footprints 1 second graph and the Footprints 1 second data export.

I'm open to suggestions.

Tom

[tlveik]

Follow up.

Around May 23rd or so I finally convinced TED support that maybe there was something wrong the the TED Pro I just purchased.  They asked me to return the ECC to check it out, so I did.  They then promptly sent me a replacement.

The replacement has been in place for a few days, and I'm happy to report that the replacement is working flawlessly.  I haven't seen a single error in the data yet.  No more missing data, no more data records out of order.  Whoo hoo!

Tom