For example if I wanted the trim in the living room to be white but upstairs I want the trim to be a light toffee color would that be alright. Is there a rule about trim having to match?
There are no real rules for paint colors. If a combination is pleasing to you then it will work. One "rule" that I use is try to keep a particular trim color confined to a room or area. The toffee trim might not look right if it runs into the white. Use corners or walls as natural breaks.
Same Trim Color Brings Unity Throughout Your Home
My feeling on this issue is that even though there is no absolute rule about trim color, trim has a unifying effect throughout your home. You can change wall colors easily, but the trim and ceilings are generally used to unify everything. When you start using different trim colors through out your house, things start to look choppy. You don't get that even flow throughout your home.