It's Done!
Note to examiners: Make sure to click the “⇠ Older” button at the bottom of this page to see posts going back to the beginning of this module
The last two weeks have once again been a whirlwind of testing, recording results and analysing all that data, and of course writing it all up into my final report and slides!
The testing went reasonably well, some very good bits where the robot was on its best behaviour and some parts so frustrating, it felt like the universe itself was taunting me. 40 test runs were completed, recorded and analysed, 20 runs for each of the two Kalman Filters I was testing. As it turns out, the speaker which I added a few weeks back for health and safety (and the convenience of knowing Bluetooth was connecting properly) gave me an additional time-saving surprise this week: It turns out that the tones it plays (alongside the loud whirr of the thrusters spinning up) are very visible when looking at an audio waveform. This turned what I thought would be a fairly arduous task analysing the time each run took, into a fairly quick and easy glance at the audio track of each run's video in some open source audio editing software (I use Tenacity, a good fork of the (in)famous Audacity). I didn't even have to extract the audio tracks first, opening the video files just worked! Check it out:
The part I've highlighted in the red box at the bottom of the image shows the length of the selected audio down to the millisecond (though I opted to record this to the nearest half second to account for differences in the recorded audio and exactly where I was clicking on each track). Recording oscillations still meant watching the video all the way through though, and there's a chunk of my report explaining the methodology of how I've counted what counts as an oscillation that I won't repeat here.
The same goes for the analysis methodology and results, this blog is designed to be a record of what I've been working on rather than a detailed explanation of the work itself, again, it's all in the report. That said, I did learn about a rather neat piece of statistical analysis that I will share the results for here, because the graphs are quite nice.
These are ECDF graphs, and they're very useful because they show what percentage of runs met what quality very intuitively. If you want to find out how fast the best 90% of runs finished, you can just trace a line on the time graph from 0.9 up to the curve and then trace it down to the value and you get ~85 seconds for the LKF and ~105 seconds for the EKF, same for the best 50% or n% and it works with any continuous metric. Since osciallations are discrete and always positive, you can use something called a Common Language Effect Score (CLES) which is less visually impressive but resulted in a useful sounding statistic that ~60% of LKF runs performed better in terms of number of oscillations than EKF runs. Obviously that's not the be-all and end-all but it is effective at getting across a message nonetheless.
Anyway that's it! Everything is submitted and in half an hour's time, the deadline that the last 12 weeks have been leading up to will have passed. I'm actually quite proud of everything I've accomplished in that time, and all I can do now is wait and hope my final grade reflects that. In the mean time, I'll be treating myself to a long nap!