Both witness data and the rest of the transaction data count toward a single limit. That limit is 4 million weight units (WU) or, equivalently, 1 million virtual bytes (vB). Traditional transaction data counts as 4 WU (1 vB) per actual data byte. Witness data in segwit transactions counts as 1 WU (0.25 vB) per data byte.

Thus, a block can include up to 1 MB of non-witness data or up to (almost) 4 MB of witness data, but not both at once. If you have 3 MB of witness data (= 0.75 MvB), there's only 0.25 MB left for non-witness data.

So in order to reach the theoretical maximum of (almost) 4 MB in a block, you need the block to consist almost entirely of witness data, with only some minimal transaction data (some 100 to 200 B).