Maintaining a balance of excitation and inhibition in cortical neurons is considered critical to healthy human brain function. Learning and memory acquisition introduce changes at excitatory synapses which disturb balance and threaten runaway excitation. A candidate mechanism for re-establishing balance involves the formation of inhibitory replicas of newly-formed excitatory connection. Here, we first discuss some requirements for the interaction of excitatory and inhibitory synaptic plasticity and show that finely tuned excitatory and subsequent inhibitory receptive fields emerge naturally in feed forward network models with only a few simple ingredients. In a second step we assess experimental evidence for selective inhibitory rebalancing in humans. We developed non-invasive tools using Magnetic Resonance Imaging (MRI) to index circuit level neural activity in the human brain and adapted a neural network model to mirror our experimental protocol: By measuring newly-formed associations we show that expression of cortical associations reduces with time despite persistence in behaviour, consistent with restoration of balance via inhibitory plasticity. To test this hypothesis, we modulated the excitation/inhibition balance by reducing cortical GABA levels with trans-cranial direct current stimulation (tDCS). Using Ultra High-Field (7T) MRI and spectroscopy, we show that reducing GABA induced a proportional increase in the expression of the cortical association. This suggests that when cortical excitatory synapses are potentiated during memory formation, the network is subsequently rebalanced by opposing and proportional changes at inhibitory synapses.