Complete System Failure In Flight

[DBRV10]

As many know, we have had a number of reports where SkyView will become unresponsive to input for a lengthy period of time. In many cases where the user did not reboot the display (buttons 1,2,5) it would remain in the unresponsive state for up to 12 minutes before recovering. Usually, functions such as autopilot remained working but the display and user I/O was useless. Not good. Furthermore, the diagnostic logs from users that experienced this event were not helpful in isolating the offending piece of code.

We created a new version that has added watchdog functionality that will reboot the display if the unresponsive state is detected (within a couple seconds). While a reboot is not ideal behavior it is far better than being unresponsive for 12 minutes or requiring the user to reboot. When this version reboots, it will log the state of memory hopefully identifying the culprit piece of software that is causing this problem.

We are working hard on a solution to the root cause which will result in another software release. Don't be surprised if we release interim versions to gather different diagnostics as we learn more.

Software Download Page

We recommend all v14 users to update to this revision (and future patches). Contact tech support if you would rather downgrade to v13.

This is why we buy Dynon. This is why I recommend and ultimately sell dozens of Dynon systems down under. And I don't actually sell them.

This is what you do not get from the big G or the likes of Asp3n or others when systems fail really badly.

Thanks a heap to Rob, Paul, Mike and Lawrence and all the others who work really hard giving us a great product.

Like flying and landing a C185/195, it is not how well you can land them, it is how well you recover from a shocker that counts! 

 

[Dynon]

As we indicated above, we have a new version of 14.0.1 up on our download page. It does not have any further improvements to the behavior yet, but we've improved the diagnostic logging capabilities further (it turned out that we didn't quite catch the events that we were looking for in the first couple of 14.0.1 logs that we received). So if you've had this problem and are willing to keep going to help us track it down, we'd love to have you upgrade.

Of course, if you want to go back to v13 in the interim, that offer remains on the table. Contact our tech support staff at support at dynonavionics dot com if you want to go down that path.

 

[dynonsupport]

We have found an interesting commonality between systems that are experiencing the unresponsive state. In 13 of 13 systems examined the software issue only appeared when it is running on particular versions of hardware. We still think that it is purely a SW issue and cannot yet explain the pattern we have noticed however the pattern is strong enough that we felt it important to communicate. With some confidence we can say:
If you are running a D1000 Touch then you should not see this problem
If you are running a D1000 serial number 6000 or higher then you should not see this problem
If you are running a D700 serial number 4000 or higher then you should not see this problem

Another way of determining if you are running the vulnerable HW/SW combination is to go into SETUP -> LOCAL DISPLAY SETUP -> DISPLAY HARDWARE INFORMATION and look for HARDWARE VERSION: 1. That is the vulnerable combination. If it says HARDWARE VERSION: 2 then you are in the above three groups. 

So, what does that mean?
1. We REALLY want to see diagnostic logs from people if they have seen this problem but fell into the above three groups because it may provide new information.
2. We would like to build confidence on this discriminator.

 

[turbopilot]

We have found an interesting commonality between systems that are experiencing the unresponsive state. In 13 of 13 systems examined the software issue only appeared when it is running on particular versions of hardware. We still think that it is purely a SW issue and cannot yet explain the pattern we have noticed however the pattern is strong enough that we felt it important to communicate. With some confidence we can say:
If you are running a D1000 Touch then you should not see this problem
If you are running a D1000 serial number 6000 or higher then you should not see this problem
If you are running a D700 serial number 4000 or higher then you should not see this problem

Another way of determining if you are running the vulnerable HW/SW combination is to go into SETUP -> LOCAL DISPLAY SETUP -> DISPLAY HARDWARE INFORMATION and look for HARDWARE VERSION: 1. That is the vulnerable combination. If it says HARDWARE VERSION: 2 then you are in the above three groups. 

So, what does that mean?
1. We REALLY want to see diagnostic logs from people if they have seen this problem but fell into the above three groups because it may provide new information.
2. We would like to build confidence on this discriminator.

Very helpful. Thanks. I have D1000 Touch in a VFR airplane so I think I will take a chance and install the 2020 GPS and move to V14. Fingers crossed.

 

[jeffa]

Good to know about the hardware. I just finished the installation of the second display yesterday. I have D1000 SN 3143 (which is the one I experienced the autopilot disconnects and freeze up on Feb 5th) on the left and D1000 Touch SN 22300 on the right. I've installed v14.0.1.3558 on both but haven't had a chance to fly. I'll put it through its paces as soon as possible.

 

[RV9APlane]

I have D1000's with serial numbers 3789 and 3790. I downloaded the new software version today and flew with it today.  I had a freeze up and an autopilot disconnect, at separate times.  I have exported a diagnostic file and am uploading it as we speak.

 

[n456ts]

would you like me to send it in?

http://www.dynonavionics.com/docs/upload.html

 

[swatson999]

May or may not be related, but is definitely something I have never seen before (300+ hours of flight on the sV-1000)...

On initial start-up today, the GPS-250 (will swap for GSP-2020 on hand next month at annual) apparently wasn't communicating with the system, as I got a GPS failure notification, no synthetic vision, and a warning message ADS-B OUT Offline - No GPS (I grabbed a screenshot, will upload tomorrow). Power cycling everything, and all was okay. (The software engineer in me says that GPS communications were out of sync or insufficient wait states on boot-up of the SV).

As I said, may or may not be related to the latest software release, but another data point. This was first flight with the very, very latest V14 software from Friday's release...

 

[airguy]

While flying today I had two interesting things happen. I installed the newest v14 software this morning, then flew for a bit and at one point on the ground, prior to start, I started my primary screen on internal battery and it did an uncommanded reboot. It came up with a System Event 3 notification, and I immediately grabbed a diagnostic file. I also got a brief partial caution that was chopped off midstream, following by a text annunciation about "GPS 1 fail" though I was unable to find any actual failure and my GPS lock on the GPS-2020 was perfect.

I will upload the diagnostic file for you to look at.

EDIT - please disregard the file uploaded this evening dated Feb 6 - I grabbed the wrong USB stick - I will upload the correct file in the morning.

 

[Garrett]

On Saturday's trip right after startup when I went to set the fuel computer to full and hit accept the system rebooted and came back with system event 3 each time while running V14.1.

 

[jvolkober]

Similar to airguy, upgraded both SV1000 and SV700 to V14.01, restarted system.  As I was working through prestart check list, the SV700 shut down and rebooted with System 3 error.  Diagnostic file sent this morning.

I flew about 45 minutes trying a half dozen times to recreate the original problem, system freeze when accessing MAP/INFO.   Was unable to recreate problem.

 

[DBRV10]

I updated a mates RV7A and logged some events so hopefully Dynon will have a pile of data to wade through in the morning.

 

[swatson999]

Screenshot of GPS/ADS-B Out failure



(The Cross Check Attitude warning was likely due to the GNS-430W not yet acquiring a fix).

 

[krw5927]

Steve, if your 430W is your ADS-B position source, you'll see the "No ADS-B Out: GPS Lost" message until the 430W acquires a fix. Then it will go away.

 

[swatson999]

Steve, if your 430W is your ADS-B position source, you'll see the "No ADS-B Out: GPS Lost" message until the 430W acquires a fix.  Then it will go away.

It is *not* my position source for ADS-B out. The GPS-250 puck (for now, until I replace it next week with the GPS-2020 puck) is, as I said above.

 

[dynonsupport]

Steve,
What we really need to see is the screen shot of the MAP page. If there was a "?" mark on it, then you are getting GPS data but have no signal lock. If there is a red X, then we are getting no data from the GPS at all.

All the messages you got are expected when SkyView has no GPS position.

It could have just been the GPS taking a bit longer to lock on than it normally does, which doesn't indicate any kind of issue. Usually it locks in 15 seconds before the screen boots, but it can take over a minute at times.

 

[swatson999]

Steve,
What we really need to see is the screen shot of the MAP page. If there was a "?" mark on it, then you are getting GPS data but have no signal lock. If there is a red X, then we are getting no data from the GPS at all.

All the messages you got are expected when SkyView has no GPS position.

It could have just been the GPS taking a bit longer to lock on than it normally does, which doesn't indicate any kind of issue. Usually it locks in 15 seconds before the screen boots, but it can take over a minute at times.

OK...if I see it again, I'll bring up the map page and grab a screenshot, and attempt to wait it out and see what happens, as well.

BTW, I only bring this up because it's a behavior I have not seen at all in nearly 400 hours of flight, but occurred immediately after upgrading to the very latest software version (and I mean *immediately prior, as in I upgraded Friday after flying, then this occurred at the beginning of the very next flight), so I get...suspicious.

 

[Dynon]

I have D1000's with serial numbers 3789 and 3790. I downloaded the new software version today and flew with it today.  I had a freeze up and an autopilot disconnect, at separate times.  I have exported a diagnostic file and am uploading it as we speak.

What exact version were you running Bruce? And did it completely stop working or did it reboot?

 

[RV9APlane]

I am running 14.0.1.3558. I didn't know it would do a self-reboot so I immediately did a 1, 2, &5 reboot as soon as I saw the screen freeze up.

 

[krw5927]

Had another in-flight lock-up last night.  Unfortunately still on V14.0 so logs won't help.

I did notice, though, that it happened either during or immediately following either syncing or changing the heading bug using my knob module.  Thinking back to my first in-flight lock-up, it's likely I was syncing or changing heading via the knob module then as well.

The first one was using AP, the second hand flying.  So there's no correlation there in my case.