Data logger made

G

ggerhardt

New Member
Joined
Nov 18, 2010
Messages
54
Rather than wait for the busy folks at Dynon to enable on-board logging, I built my own serial data logger.  I describe it on my build site (http://thegerhardts.com/velocity), but basically I used the little OpenLog board from SparkFun along with a MAX232 level shifter.  The OpenLog board accepts a microSD card, so after a flight, you just need to pull out the card and stick it in your computer to get the log file.

If you turn on all logging, the log file gets a bit confusing as each line needs to be parsed differently (i.e. EMS, ADARHS, SYSTEM data are on different lines).  I made and excel macro that parses the log and puts each data into a separate worksheet.  I can post the macro here if anyone is interested in it.

Geoff
 
G

ggerhardt

New Member
Joined
Nov 18, 2010
Messages
54
Here it is.  If you're not sure how to import a macro into Excel, here's a video on it (http://www.youtube.com/watch?v=YqJTD-_Hcmg).

To use the macro, just run it.  It will open up a file open dialog box, pick your log file and it should load it then parse it.  I'm not all that great at writing excel macros, so its nothing special.  It doesn't put headers on the columns, so you'll need to go to the Skyview manual to figure out what means what.  I'd appreciate if anyone knows excel VB better that they could clean it up and repost it.  It does the basic job of parsing the data steam into the separate bits, tho.

Oops, the forum doesn't allow text file attachments.  I've pasted the macro below.  Cut it out, paste it either into your VB module of excel or into a notepad file and follow the above video to import it into excel.

Geoff

-------------------------
Sub DynonDataParser()
'
' ADAHRSData Macro
'
' Keyboard Shortcut: Ctrl+a
'
    Application.DisplayAlerts = False
    Application.ScreenUpdating = False
   
    FName = Application.GetOpenFilename("Dyon log files (*.txt), *.txt")
    Workbooks.OpenText Filename:=FName, _
        Origin:=xlWindows, StartRow:=2, DataType:=xlDelimited, TextQualifier:= _
        xlDoubleQuote, ConsecutiveDelimiter:=False, Tab:=True, Semicolon:=False, _
        Comma:=False, Space:=False, Other:=False, FieldInfo:=Array(1, 1), _
        TrailingMinusNumbers:=True
   
    Dim lastRow, lastRawRow As Long
    Dim RawData, ADAHRSws, SYSTEMws, EMSws As Worksheet
    Dim PctDone As Single
    
    Set RawData = ActiveWorkbook.Sheets(1)
    RawData.Name = "RAW DATA"
   
    ActiveWorkbook.Sheets.Add
    Set ADAHRSws = ActiveWorkbook.Sheets(1)
    ADAHRSws.Name = "ADAHRS"
   
    ActiveWorkbook.Sheets.Add
    Set SYSTEMws = ActiveWorkbook.Sheets(1)
    SYSTEMws.Name = "SYSTEM"
   
    ActiveWorkbook.Sheets.Add
    Set EMSws = ActiveWorkbook.Sheets(1)
    EMSws.Name = "EMS"
   
    Dim RowNum As Integer
   
    RawData.Activate
    lastRawRow = Range("A65536").End(xlUp).Row
    For RowNum = 1 To lastRawRow
        'ADAHRS data parsing
        RawData.Activate
        If Mid(Cells(RowNum, 1), 2, 1) = "1" Then
            RawData.Activate
            Cells(RowNum, 1).TextToColumns DataType:=xlFixedWidth, _
                FieldInfo:=Array(Array(0, 1), Array(1, 1), Array(2, 1), Array(3, 1), Array(5, 1), Array _
                (7, 1), Array(9, 1), Array(11, 1), Array(15, 1), Array(20, 1), Array(23, 1), Array(27, 1), _
                Array(28, 1), Array(33, 1), Array(34, 1), Array(37, 1), Array(40, 1), Array(43, 1), Array( _
                45, 1), Array(49, 1), Array(52, 1), Array(56, 1), Array(59, 1), Array(60, 1), Array(65, 1), _
                Array(68, 1), Array(70, 1), Array(72, 1)), TrailingMinusNumbers:=True
            Rows(RowNum & ":" & RowNum).Select
            Selection.Copy
            ADAHRSws.Select
            lastRow = 1 + ADAHRSws.Range("A" & Rows.Count).End(xlUp).Row
            ADAHRSws.Rows(lastRow & ":" & lastRow).Select
            ADAHRSws.Paste
        End If
       
        RawData.Activate
        'SYSTEM data parsing
        If Mid(Cells(RowNum, 1), 2, 1) = "2" Then
            RawData.Activate
            Cells(RowNum, 1).TextToColumns DataType:=xlFixedWidth, _
                FieldInfo:=Array(Array(0, 1), Array(1, 1), Array(2, 1), Array(3, 1), Array(5, 1), Array _
                (7, 1), Array(9, 1), Array(11, 1), Array(14, 1), Array(15, 1), Array(19, 1), Array(23, 1), _
                Array(24, 1), Array(27, 1), Array(30, 1), Array(31, 1), Array(32, 1), Array(34, 1), Array( _
                35, 1), Array(37, 1), Array(38, 1), Array(40, 1), Array(41, 1), Array(42, 1), Array(43, 1), _
                Array(45, 1), Array(46, 1), Array(48, 1), Array(49, 1), Array(51, 1), Array(52, 1), Array( _
                53, 1), Array(54, 1), Array(58, 1), Array(60, 1)), TrailingMinusNumbers:=True
            Rows(RowNum & ":" & RowNum).Select
            Selection.Copy
            SYSTEMws.Select
            lastRow = 1 + SYSTEMws.Range("A" & Rows.Count).End(xlUp).Row
            SYSTEMws.Rows(lastRow & ":" & lastRow).Select
            SYSTEMws.Paste
        End If

        RawData.Activate
        'EMS data parsing
        If Mid(Cells(RowNum, 1), 2, 1) = "3" Then
            RawData.Activate
            Cells(RowNum, 1).TextToColumns DataType:=xlFixedWidth, _
                FieldInfo:=Array(Array(0, 1), Array(1, 1), Array(2, 1), Array(3, 1), Array(5, 1), Array _
                (7, 1), Array(9, 1), Array(11, 1), Array(14, 1), Array(18, 1), Array(22, 1), Array(26, 1), _
                Array(29, 1), Array(32, 1), Array(35, 1), Array(38, 1), Array(41, 1), Array(44, 1), Array( _
                47, 1), Array(50, 1), Array(53, 1), Array(57, 1), Array(62, 1), Array(67, 1), Array(71, 1), _
                Array(75, 1), Array(79, 1), Array(83, 1), Array(87, 1), Array(91, 1), Array(95, 1), Array( _
                99, 1), Array(103, 1), Array(107, 1), Array(111, 1), Array(115, 1), Array(119, 1), Array( _
                123, 1), Array(129, 1), Array(134, 1), Array(135, 1), Array(141, 1), Array(146, 1), Array( _
                147, 1), Array(183, 1), Array(188, 1), Array(189, 1), Array(194, 1), Array(195, 1), Array( _
                217, 1), Array(219, 1)), TrailingMinusNumbers:=True
            Rows(RowNum & ":" & RowNum).Select
            Selection.Copy
            EMSws.Select
            lastRow = 1 + EMSws.Range("A" & Rows.Count).End(xlUp).Row
            EMSws.Rows(lastRow & ":" & lastRow).Select
            EMSws.Paste
        End If
       
        PctDone = RowNum / lastRawRow

    Next
   
    Unload UserForm1
    ADAHRSws.Activate
    ADAHRSws.Columns("A:C").Select
    Selection.Delete
    ADAHRSws.Rows("1:1").Select
   
    SYSTEMws.Activate
    SYSTEMws.Columns("A:C").Select
    Selection.Delete
    SYSTEMws.Rows("1:1").Select
   
    EMSws.Activate
    EMSws.Columns("A:C").Select
    Selection.Delete
    EMSws.Rows("1:1").Select
    RawData.Delete
   
    ADAHRSws.Activate
   
    Application.DisplayAlerts = True
    Application.ScreenUpdating = True
   
End Sub
 
lolachampcar

lolachampcar

New Member
Joined
Jul 17, 2011
Messages
249
I used the same sparkfun board and a FET to do the RS-232 to 3.3V level conversion.  I wrote a parser to generate a log data file that can be viewed by data viewing software (I use MegaLogViewer from the Megasquirt guys).  Here is some data - http://www.lolachampcar.com/MX/TestingTuning/testing_and_tuning.htm

I'm doing EMS and ADAHARS data only in my parser but am happy to drop the exe on here (and the C souce) along with an example .log output file if it would be helpful....
 
G

ggerhardt

New Member
Joined
Nov 18, 2010
Messages
54
I don't think you can attach exe files to posts here, but I'd like to have a look at it. Can you send it (C source, too) to me via email (geoff dot gerhardt at gmail dot com)?

Thanks.

Geoff
 
lolachampcar

lolachampcar

New Member
Joined
Jul 17, 2011
Messages
249
This zip is a Visual Studio 2005 project folder.  The .exe is in the bin/release folder

Sample logged data
!1114043300+076+00160710000-00055+000+03+1099-002+300000261+01696XXXXX21
!3114043300XXXXXXX00000000XXX000000XXXXXXXXX170133XXX-0270017700140+029+030+029+030+029+030+030+030+029+030+030+031XXXXXXXX+0027T+0115GZZZZZZ+0014PZZZZZZZZZZZZZZZZZZZZZZZZXXXXXXXXXXXX+0013P+0287CZZZZZZZZZZZZZZZZZZZZZZ9D
!1114043404+076+00160710000-00056+000+03+1099-006+300000261+01695XXXXX2A
!3114043407001+02900000000300000000014XXXXXX170133XXX-0270017700140+029+030+029+030+029+030+030+030+030+030+030+031XXXXXXXX+0027T+0115GZZZZZZ+0014PZZZZZZZZZZZZZZZZZZZZZZZZXXXXXXXXXXXX+0013P+0287CZZZZZZZZZZZZZZZZZZZZZZA4
!1114043415+076+00160710000-00057+000+03+1099-008+300000261+01695XXXXX2F
!3114043202001+02900000000300000000014XXXXXX170133XXX-0280017700140+029+030+029+030+029+030+030+030+029+030+030+031XXXXXXXX+0027T+0115GZZZZZZ+0014PZZZZZZZZZZZZZZZZZZZZZZZZXXXXXXXXXXXX+0013P+0287CZZZZZZZZZZZZZZZZZZZZZZA6
!1114043210+076+00160710000-00057+000+03+1099-005+300000261+01695XXXXX25
!3114043213001XXXX00000000300000000014XXXXXX170133XXX-0280017700140+029+030+029+030+029+030+030+030+029+030+030+031XXXXXXXX+0027T+0115GZZZZZZ+0014PZZZZZZZZZZZZZZZZZZZZZZZZXXXXXXXXXXXX+0013PXXXXXXZZZZZZZZZZZZZZZZZZZZZZ13
!1114043305+076+00160710000-00058+000+03+1099-007+300000261+01695XXXXX2D
!3114043308001XXXX00000000299000000014XXXXXX170133XXX-0280017700140+029+030+029+030+029+030+030+030+029+030+030+031XXXXXXXX+0027T+0115GZZZZZZ+0014PZZZZZZZZZZZZZZZZZZZZZZZZXXXXXXXXXXXX+0014PXXXXXXZZZZZZZZZZZZZZZZZZZZZZ2A
!1114043300+076+00160710000-00059+000+03+1099-006+300000261+01691XXXXX24
!3114043403001+02900000000299000000014XXXXXX170133XXX-0280017700140+029+030+029+031+029+030+030+031+029+031+030+031XXXXXXXX+0027T+0115GZZZZZZ+0014PZZZZZZZZZZZZZZZZZZZZZZZZXXXXXXXXXXXX+0013P+0288CZZZZZZZZZZZZZZZZZZZZZZBE
!1114043411+076+00160710000-00061+000+03+1099-008+300000261+01691XXXXX22

which is converted to
Time,Pitch,Roll,Yaw,IAS,PAlt,TurnRate,LatG,VertG,AoA,VSpeed,OAT,TAS,Baro,DA,WS,WD,OP,OT,RPML,RPMR,MAP,FuelFlow,FP,FuelRem,V,Amps,Hobbs,Tach,CHT6,EGT6,CHT5,EGT5,CHT4,EGT4,CHT2,EGT2,CHT3,EGT3,CHT1,EGT1,ElTrim,AcroF
0.00, 0.0, 0.0, 0, 0.0, 0, 0.0, 0.0, 0.0, 0, 0, 0, 0.0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 0.0, 0, 0.0, 0.0, 0.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 0.0
0.00, 0.0, 0.0, 0, 0.0, 0, 0.0, 0.0, 0.0, 0, 0, 0, 0.0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 0.0, 0, 0.0, 0.0, 0.0, 0, 0, 450, 1500, 450, 1500, 450, 1500, 450, 1500, 450, 1500, 450, 1500, 0.0, 0.0
0.00, 7.6, 1.6, 71, 0.0, -55, 0.0, 0.3, 1.0, 99, -20, 86, 0.0, 26.1, 1696, 0, 0, 4440, 8024, 0, 0, 34.0, 0.0, 4440, 17.0, 13.3, -2.7, 17.7, 14, 84.2, 86, 84.2, 86, 84.2, 86, 86, 86, 84.2, 86, 86, 87.8, 2.7, 11.5
0.02, 7.6, 1.6, 71, 0.0, -56, 0.0, 0.3, 1.0, 99, -60, 86, 0.0, 26.1, 1695, 0, 0, 1, 84.2, 0, 0, 30.0, 0.0, 14, 17.0, 13.3, -2.7, 17.7, 14, 84.2, 86, 84.2, 86, 84.2, 86, 86, 86, 86, 86, 86, 87.8, 2.7, 11.5
0.05, 7.6, 1.6, 71, 0.0, -57, 0.0, 0.3, 1.0, 99, -80, 86, 0.0, 26.1, 1695, 0, 0, 1, 84.2, 0, 0, 30.0, 0.0, 14, 17.0, 13.3, -2.8, 17.7, 14, 84.2, 86, 84.2, 86, 84.2, 86, 86, 86, 84.2, 86, 86, 87.8, 2.7, 11.5
0.07, 7.6, 1.6, 71, 0.0, -57, 0.0, 0.3, 1.0, 99, -50, 86, 0.0, 26.1, 1695, 0, 0, 1, 8024, 0, 0, 30.0, 0.0, 14, 17.0, 13.3, -2.8, 17.7, 14, 84.2, 86, 84.2, 86, 84.2, 86, 86, 86, 84.2, 86, 86, 87.8, 2.7, 11.5
0.09, 7.6, 1.6, 71, 0.0, -58, 0.0, 0.3, 1.0, 99, -70, 86, 0.0, 26.1, 1695, 0, 0, 1, 8024, 0, 0, 29.9, 0.0, 14, 17.0, 13.3, -2.8, 17.7, 14, 84.2, 86, 84.2, 86, 84.2, 86, 86, 86, 84.2, 86, 86, 87.8, 2.7, 11.5
0.11, 7.6, 1.6, 71, 0.0, -59, 0.0, 0.3, 1.0, 99, -60, 86, 0.0, 26.1, 1691, 0, 0, 1, 84.2, 0, 0, 29.9, 0.0, 14, 17.0, 13.3, -2.8, 17.7, 14, 84.2, 86, 84.2, 87.8, 84.2, 86, 86, 87.8, 84.2, 87.8, 86, 87.8, 2.7, 11.5
0.14, 7.6, 1.6, 71, 0.0, -61, 0.0, 0.3, 1.0, 99, -80, 86, 0.0, 26.1, 1691, 0, 0, 1, 84.2, 0, 0, 29.9, 0.0, 14, 17.0, 13.3, -2.8, 17.7, 14, 84.2, 86, 84.2, 87.8, 84.2, 86, 86, 87.8, 84.2, 87.8, 86, 87.8, 2.7, 11.5
0.16, 7.6, 1.6, 71, 0.0, -62, 0.0, 0.3, 1.0, 99, -70, 86, 0.0, 26.1, 1691, 0, 0, 1, 84.2, 0, 0, 30.0, 0.0, 14, 17.0, 13.3, -2.8, 17.7, 14, 84.2, 86, 84.2, 86, 84.2, 86, 86, 86, 84.2, 86, 86, 87.8, 2.7, 11.5
0.18, 7.6, 1.6, 71, 0.0, -62, 0.1, 0.3, 1.0, 99, -70, 86, 0.0, 26.1, 1691, 0, 0, 1, 84.2, 0, 0, 30.0, 0.0, 14, 17.0, 13.3, -2.8, 17.7, 14, 84.2, 86, 84.2, 87.8, 84.2, 86, 86, 87.8, 84.2, 87.8, 86, 87.8, 2.7, 11.5
0.21, 7.6, 1.6, 71, 0.0, -63, 0.1, 0.3, 1.0, 99, -90, 86, 0.0, 26.1, 1686, 0, 0, 1, 84.2, 0, 0, 30.0, 0.0, 14, 17.0, 13.3, -2.8, 17.7, 14, 84.2, 86, 84.2, 87.8, 84.2, 86, 86, 87.8, 84.2, 87.8, 86, 87.8, 2.7, 11.5
0.23, 7.6, 1.6, 71, 0.0, -66, 0.0, 0.3, 1.0, 99, -120, 86, 0.0, 26.1, 1686, 0, 0, 1, 84.2, 0, 0, 30.0, 0.0, 14, 17.0, 13.3, -2.9, 17.7, 14, 84.2, 86, 84.2, 87.8, 84.2, 86, 86, 87.8, 84.2, 87.8, 86, 87.8, 2.7, 11.5
0.25, 7.6, 1.6, 71, 0.0, -67, 0.0, 0.3, 1.0, 99, -120, 86, 0.0, 26.1, 1686, 0, 0, 1, 84.2, 0, 0, 29.9, 0.0, 14, 17.0, 13.3, -2.9, 17.7, 14, 84.2, 86, 84.2, 87.8, 84.2, 86, 86, 87.8, 84.2, 87.8, 86, 87.8, 2.7, 11.5
0.27, 7.6, 1.5, 71, 0.0, -68, 0.0, 0.3, 1.0, 99, -90, 86, 0.0, 26.1, 1686, 0, 0, 1, 84.2, 0, 0, 30.0, 0.0, 14, 17.0, 13.3, -2.9, 17.7, 14, 84.2, 86, 84.2, 87.8, 84.2, 86, 86, 87.8, 84.2, 87.8, 86, 87.8, 2.7, 11.5


Hope this helps
 

Attachments

  • Skyview_Utility.ziP
    83.5 KB · Views: 144
B

Bubblehead

Member
Joined
Oct 3, 2011
Messages
37
I have tried to make a data logger like Geoff's but used a different shifter.  No luck on getting data.  The lights on the shifter show data being received but no lights on OpenLog showing data received.  Here is my wiring schematic.  I included a 7805 voltage regulator so I could use voltage from either ships power or the 5v from Skyview. 
ScreenHunter01-Sep-24-1703-M.gif


Does anyone have any ideas on why I am not receiving data? I have a config file on the card, and formatted the card to Fat32.
 
G

ggerhardt

New Member
Joined
Nov 18, 2010
Messages
54
Stupid question, but have you turned on logging for that serial port in the Skyview setup?

Geoff
 
B

Bubblehead

Member
Joined
Oct 3, 2011
Messages
37
Geoff - not a stupid question.  Often the problem is something simple.  In this case I do have SkyView configured to output EMS data on that port. and am connecting to the "TX" wire.  That is confirmed by the shifter showing data being received.  I will verify that Skyview is set up correctly when I go back to the hanger tonight.

I also hooked up a serial connection to my laptop but did not have any luck with a terminal emulation program trying to receive data.

Electronics is definitely just a sideline for me, especially digital communications.  I've taken electronics classes and built electronics kits (receivers, R/C servos etc.) but it does not come as easily to me as other things.  I do enjoy it though, which is one reason I am trying to get this to work.  I can learn a lot once I get this device working.

I ordered a MAX232 yesterday and plan on changing to the same shifter as you used.

Does the circuit look right?
 
G

ggerhardt

New Member
Joined
Nov 18, 2010
Messages
54
The circuit looks right, but it seems you're having an issue a the level shifter, tho, and that part of the circuit is not labeled, tho (what chip you're using). MAX232 is the standard to use for this type of application, so I think you'll be fine once you get that in there. Strange that you can't get anything with your PC, tho. I never tried it, but it should work ok.

Geoff
 
B

Bubblehead

Member
Joined
Oct 3, 2011
Messages
37
Here's a link to the shifter I am using.

https://www.sparkfun.com/products/8780

Here's the description from the website:
"Description: The smallest and easiest to use serial conversion circuit on the market! This board has one purpose in life - to convert RS232 to TTL and vice versa (TX and RX). This will allow a microcontroller to communicate with a computer. Shifter SMD is powered from the target application and can run at any voltage! That's right - power the board at 5V and the unit will convert RS232 to 5V TTL. Power the board at 2.8V and the Shifter board will convert RS232 to 2.8V CMOS TTL. Includes two indicator LEDs for TX and RX. Runs from 300bps up to 115200bps."
 
B

Bubblehead

Member
Joined
Oct 3, 2011
Messages
37
Here's a picture of the OpenLog and shifter on the breadboard.  The voltage regulator is in-line with the cable from the SkyView.

breadboarded-circuit-S.jpg
 
B

Bubblehead

Member
Joined
Oct 3, 2011
Messages
37
I think I just found the problem.  I had hooked up the data line from Skyview to the TTL RX pad not the RS-232 RX pad on the shifter.  Unfortunately my soldering iron is at the hanger but I will be up early to go to the hanger, move the wire, add a ground from RS-232 ground to the other grounds and plug it into the airplane and see if I get data. Very exciting. I feel like a 7 year old on Christmas Eve.
 
B

Bubblehead

Member
Joined
Oct 3, 2011
Messages
37
Yippee Skippy! Changed one wire, added a ground to the RS gnd pad and recorded data! Now I need to figure out how to make sense of it! Here is what one line of data looks like.

!3112074208000+02900000000292000000011140162263123XXXXXXX0703800674XXXXXXXXXXXXXXXX+030+029+030+029+030+029+030+029XXXXXXXXXXXXXXXXXXXXZZZZZZ+0011P+4998VZZZZZZZZZZZZZZZZZZ+0140G+0162G+0000P+0287CXXXXXXZZZZZZZZZZZZZZZZ24

That looks a lot like lolachampcar's data. I am just recording EMS data right now and lolachampcar's utility is apparently looking for ADAHARS as well as EMS data. I am hoping he (or she) can modify the utility to handle just EMS data. Alternatively I will go to the hanger tonight, change the setup in Skyview and record some more data.
 
D

DarrellK

New Member
Joined
Jun 11, 2011
Messages
21
Location
Waukesha Wisconsin
I would like to build one of these too. Looks like a lot of fun. I just wish we knew how long it will be before Dynon adds this function into the software.
 
D

dynonsupport

Dynon Technical Support
Staff member
Joined
Mar 23, 2005
Messages
13,226
Data logging is something that is actively being coded at Dynon right now. That doesn't promise a release date, but after ADS-B weather and traffic it's our highest feature priority.
 
B

Bubblehead

Member
Joined
Oct 3, 2011
Messages
37
I was able to record ADAHRS plus EMS data and convert it using lolachampcar's utility. When I get time this weekend I will correct the schematic, put a bill of material on it and write some instructions. Hopefully that will help people get some data until things come down the pike from Dynon.
 
B

Bubblehead

Member
Joined
Oct 3, 2011
Messages
37
Here is the corrected schematic with a parts list.  I also attached it so you can download it.  Set Skyview data output to ADAHARS and EMS and use the utility that lolachampcar posted earlier to convert the data, then inport it into Excel as a comma delimited file.
 

Attachments

  • schematic.JPG
    schematic.JPG
    92.2 KB · Views: 130
lolachampcar

lolachampcar

New Member
Joined
Jul 17, 2011
Messages
249
I was able to record ADAHRS plus EMS data and convert it using lolachampcar's utility. When I get time this weekend I will correct the schematic, put a bill of material on it and write some instructions. Hopefully that will help people get some data until things come down the pike from Dynon.
Click to expand...

I've found some mistakes and made a few changes. I'll post the new code shortly.

BTW, one of the spark fun serial data logging boards together with a single fet transistor and a single resistor are all that is needed to convert RS-232 level serial to the logic level serial used by the board (3.3V if memory serves me). I can gen up a simple schematic if it will help.
 
You must log in or register to reply here.
Top