Adaptivity has been proposed as a way of optimizing the game experience, but can games with adaptive leveling lead to learning gains? We present results from a study evaluating two versions of a digital game called Fish Force—an adaptive version and a nonadaptive version—as compared with a control game. One hundred forty-four children (aged 5–7) were randomly assigned to one of three conditions: adaptive, nonadaptive, or control. Analyses revealed that children in the nonadaptive condition performed statistically significantly better on a physics knowledge posttest as compared with children in the control, when controlling for age, gender, and low-income status. There was no significant difference between the adaptive and nonadaptive conditions. We also examined differences in self-reported engagement between the two treatment groups. Children in the nonadaptive group reported significantly higher levels of engagement than the adaptive group. We discuss implications for designing adaptive game systems.