By default Oracle does case sensitive sorting, which means that ascending order goes from A-Z then a-z. This is reflected in APEX when creating reports and sorting on them.

They're various workarounds to enable case insensitive sorting. The simplest is to change the entire application by modifying the following setting in Shared Components > Globalization > Character Value Comparison to BINARY_CI:

Once set, all your reports will be sorted case insensitive. The first report now looks like this:

Thanks to Jorge Rimblas for showing me this!