Neo at Night: Not Recommended

DJI says there's no Night Mode for flying the Neo. It definitely cannot use any of the AI quick shots. But with an RC, it seems like you should be able to fly it. The camera won't do well, but it should work, right?
Well, when I walked outside and saw the end of a sunset, I thought I'd throw this easy to fly drone up and see what happened. It's an empty parking lot. Just wanted to take it straight up and back down.
Things did not go as planned.
I don't understand why the drone drifts in the dark. Inside, when trying to use AI in a darker room, it doesn't just hover, it starts dropping and drifting to the side. Now, outside at night, I see similar behavior. I was able to use the remote to overcome the drift, but it was a fight.