Windguard method including data filtering

[reos-rcrozier]

We noticed that windguard also have a power-to-power comparison which is different from the smart blade method. I think there is some merit in implementing both methods and seeing how they compare. The Windguard method seems more intuitive.

However, in Windguard's document they identify some data filtering steps which I think will be useful for both methods. Therefore I began working on this data filtering code in this branch based on these recommendations. The recommendations are:

• Test Turbine and Reference Turbine operational, filtering by status code or e.g. by RPM signal
• Test Turbine and Reference Turbine not curtailed, filtering by status signal
• No blade icing, filtering by temperature
• Test Turbine and Reference Turbine free of wakes is often preferable, but no necessary condition
• In Testing Period: use only wind direction ranges and power ranges also covered by Training Period

@jecheniq @reos-otulloch @Visgaard

[reos-rcrozier]

I have also in this branch now moved to using Matlab packages. I chose the package name windp2p so the functions in the package a re called like

windp2p.filterData
windp2p.findWakeFreeConditions

I have also split out the parse_pv_pairs function to it's own file instead of being a subfunction. For this project it must now also be called with the following syntax windp2p.parse_pv_pairs. You will see I updated the code to do this.

[Visgaard]

Hey Richard. That looks good to me. I've been working a little bit on the syncing of time-series and I found a way to do it with a set tolerance.

`
TimeTolerance = datenum(seconds(options.TimeTolerance));
[iControlPre, iTest] = ismembertol( datenum(Data.ControlWTPre.Time), ...
datenum(Data.TestWTPre.Time), ...
TimeTolerance, ...
'DataScale', 1);

iTest(iTest==0) = [];
iTestPre = false(size(iTest,1),1);
iTestPre(iTest) = true;

syncedTimeTestPre = Data.TestWTPre.Time(iTestPre);
syncedTimeControlPre = Data.ControlWTPre.Time(iControlPre);
`
It is of course enough to use iControlPrefind the shared synced time vector, but in case you want the index for both the control and test turbine, you can access them with iControlPre and iTestPre respectively.

In cases where Data.ControlWTPre.Time and Data.TestWTPre.Time are not of the same size, it's probably a good idea to choose the longest vector as the first input to the ismembertol function.

Let me know what you think :-)

edit: mixed up a few variables

[reos-rcrozier]

@Visgaard Yes, I've been thinking about this too, however, I think the right approach is actually more likely to be to interpolate one of the data sets to match the other. This will probably have to be done in blocks, because there will be large sections of data removed.

There is a built-in matlab function to synchronize data, with the option of interpolation. I think we may be able to use this.

The only problem is big gaps in either data series will produce bad results from interpolation, but we could identify the big gaps and remove them from the harmonised time series after synchronisation with something like the following.

% calculate durations between time steps in both data sets, something like

time_diff_ControlPre = diff (Data.ControlWTPre.Time);
time_diff_TestPre = diff (Data.ControlWTPre.Time);

% identify the data where the differences are below a chosen threshold, 
% e.g. 5 x the nominal sample period or something
sample_threshold_ControlPre = time_diff_ControlPre < options.GapThreshold
sample_threshold_TestPre = time_diff_ControlPre < options.GapThreshold

Next we could find the start and end of the continuous blocks above the threshold (I googled this a bit and there are some solutions we can use), and lets say we end up with a three column matrix with the start index, end index and size of the good blocks for each data set. Then we can use this information to remove 'bad' chunks from the synchronized data.

[reos-rcrozier]

I suppose the above is simpler if we synchronize the time series before we do any filtering on other criteria, so maybe that is the solutions. I think we would still want to check and warn about big gaps though.

[Visgaard]

I think I understand where you are getting at - correct me if I'm wrong. By using the ismembertol functionalty, we may have data from one turbine that is up to "TimeTolerance" seconds delayed from the other turbine. That's undesirable since dynamics can change significantly in this time. What you are doing is synchronizing the time to nearest seconds, and then linearly interpolating the signals of interest such as active power. Have I understood that correct?

If that's the case then I agree with you. I think that's the way to go. I believe we can handle the issue of big gaps.

On a different note - how do you make the background gray like that when you comment? I tried the "Add code <ctrl+e>" but that didn't work for me :)

[reos-rcrozier]

Yes, that's right, we should interpolate to find the values at the exact same time stamp. synchronize does this for us, but it does mean using the timetable object for the data instead of a simple structure, maybe this is fine, I haven't used it much before.

The main issue will be when there are big gaps in either time series, e.g. because the internet went down or something. The interpolated values in these sections will be nonsense, so we would need to strip them out.

For adding a block of code I just type it out, then highlight it and click on the Add code button! Or you can use backticks
three top and bottom of code blocks, or one either side of the words for inline

[Visgaard]

That makes perfect sense to me. I don't have experience with the timetable object either. But I think we need to have a structure for the data that is "light". Just 3 days of 1s log is 259200 data points, which is quite a lot. I've been using cell arrays for the data analysis that I do at work, but I find it's too slow when dimensions are high.

Test Code

[reos-rcrozier]

I'm going to go ahead and merge this branch since it doesn't interfere with anyone.