We study here the emerging phenomena of an auto-associative binary neural network where, with a probability $c$ we added a new random contribution to the traditional Hebbian synaptic weights. This new term, randomly taken from a Gaussian bimodal distribution, balances the population of synapses in our network so that we have $80\%-20\%$ relation in E/I population ratio, imitating the balance observed in mammals cortex. For some regions of the relevant parameters, our system depicts standard memory and non-memory attractors as in the Hopfield model, but as $c$ increases the system underlies a reentrant first order transition when the level of the underlying noise decreases below a certain value. This results in a low noise phase where memory disappear and where a frustrated state similar to the spin-glass phase emerges. However and contrary to what it is expected in Hopfield model, this low noise frustrated state occurs even in the limit of the loading parameter $\alpha \to 0$ which seems intriguing. Attending to mean firing rate, we observed that the phase of frustrated memory results in two states of non-vanishing activity, one with high activity or Up state and the other with low activity or Down state. Using linear stability analysis we have studied the regions in the space of relevant parameters where different steady states are locally stable. We also computed region where metastability emerges. We have studied in deep our system, both theoritecally using standard mean-field techniques, and by means of Montecarlo simulations, obtaining an almost perfect agreement between theory and simulations. The model can be useful to explain the emergence of Up and Down states similar to those in mammalian cortex and to explore the conditions to induce transitions among them.