After a few months of searching for an definitive answer to this topic, I've reached the conclusion that my original assumption would be correct. This page describes how to determine the midpoint of a standard PCM audio signal:
https://gist.github.com/endolith/e8597a58bcd11a6462f33fa8eb75c43d
For example an 8-bit signed PCM signal has these ranges:
Min: -128
Max: 128
Midpoint: 0
As to why the signal I'm getting from my soundcard sits at -128 when there is no sound, I'm going to assume that this is related to a driver problem or could be that this particular piece of hardware does not follow the PCM standard.
Converting to the logarithmic scale in my understanding is not related to this issue because you should be able to normalize the signal in time-domain even though eventually you will most likely need to convert it to the log scale if you are doing anything in the frequency domain (e.g. FFT).
If anyone has any input, please feel free to add it.