I agree with Recovery Man
I had a similar problem when by accident I caused a short on the battery and it fried one of the diodes in alternator and allowed the alternator to bleed charge from the battery when the engine was not running, at first it would take a couple of days to drain but in a very short time I was having trouble every morning, I ended up doing what your doing and disconnect the battery when ever I left it for a while.
I found the problem by putting a meter in line with the battery measuring the current drawn and disconnecting things until I found what was causing the problem.
Removing the small wires from the back of the alternator was all it needed to stop the charge drain.