The Level mode's goals are to zero your vertical speed and make the roll angle level.
Does the autopilot status are say "straight" and "level" in the upper left?
Do other modes seem to work correctly?
Can you take a video of the behavior you're seeing and email it to us at support at dynonavionics dot com so that we can help you further?