The primary quantity in statistical mechanics is the probability $P_i$ of finding the system in eigenstate ii. Once we know the probability of each eigenstate for any given state, we will be able to compute every thermodynamic property of the system.
The approach we are going to use is to state that the probabilities are those which maximize the fairness (or minimize the bias). So we need to define a fairness function $\mathcal{F}$ that we can maximize. First, let's talk about some properties the fairness function should satisfy.
1. it should be continuous 2. it should be symmetric $$\mathcal{F}(P_1,P_2,P_3,\ldots) = \mathcal{F}(P_3,P_2,P_1,\ldots)$$ 3. it should be minimum when $P_2=P_3=\ldots = 0$ and $P_1 = 1$ $$\mathcal{F}(1,0,0,\ldots) = \text{minimum}$$ 4. it should be maximum when $P_1 = P_2 = P_3 = …$ $$\mathcal{F}(P,P,P,\ldots) = \text{maximum}$$ 5. Addition rule if I have two uncorrelated systems, then their fairness should add (extensivity!!!). This corresponds to the following: $$\mathcal{F}(P_A,P_B) + \mathcal{F}(P_1,P_2,P_3) = \mathcal{F}(P_AP_1, P_AP_2,P_AP_3,P_BP_1,P_BP_2,P_BP_3)$$ There aren't many functions which satisfies all these rules! $$\text{Fairness} = -k\sum_i^{\text{all states}} P_i \ln P_i$$ This particular function satisfies all these constraints. It is continuous, symmetric, minimum when maximally unfair and maximum when maximally fair. Continuous and symmetric are reasonably obvious.
Let's show $\mathcal{F}$ is minimum when maximally unfair. This is when one $P_i=1$ and the rest are zero. $$\lim_{P\rightarrow 0} P\ln P = 0 \times \infty = \lim_{P\rightarrow 0} \frac{\ln P}{\frac1{P}} = \lim_{P\rightarrow 0} \frac{\frac1P}{-\frac1{P^2}} = 0$$ We next consider the contribution for the $P=1$, but that's easy, since $\ln 1=0$, so that term is also zero. Since $P\ln P$ can never be positive for $0≤P≤1$, we can see that the maximally unfair situation has minimum fairness, as it should.
Next, let's consider the maximally-fair situation, where $P_1=P_2=\cdots=\frac1{N}$. To demonstrate that this is maximum fairness is more tricky, and will be addressed later, when we will go about maximizing the fairness.