Ha, I'm the same way.
I use my gortex bivy and think it's totally awesome.
In the winter, I have a good -30 civvy bag. It's lighter, compresses way smaller and has a hood. I take that with me any time it's cooler than -5 or so.

Summer, I have a cheapo civvy bag that my mommy covered in some cadpat material she had and it works just great for this time of year as long as I'm in my bivy.
I ALWAYS have my ranger blanket stuffed in to my little pillow case (also made by mom!) so if I ever get too chilly at night, I can pull that out and wrap up in that too and use my coat or whatever as my pillow.

The military bags are plenty warm. I can't recall ever being cold in them, unless it was user error. I just go with the civvy bags because they compress down and weigh so much less.

I also have no clue where my liner is... I would much rather be wrapped up in fleece too, and it's great to keep for those nights where it's not hot or cold, makes things just right on it's own!

Also, if you go with the military stuff, I never tie my liner in. All it will do is become tangled and it takes you 20 minutes to get in the bag and it all sorted out. I just leave it in there loosely and I can slide right in when I go down for some sleep.

I think my favourite piece of aftermarket kit that is comfort related would be my "softie". so warm and cozy!