We have shown in previous work that LSTM networks are effective at predicting blood glucose levels in patients with type I diabetes, outperforming human experts and an SVR model trained with features computed by manually engineered physiological models. In this paper we present the results of a much larger set of experiments on real and synthetic datasets in what-if, agnostic, and inertial scenarios. Experiments on a more recent real-patient dataset, which we are releasing to the research community, demonstrate that LSTMs are robust to noise and can easily incorporate additional features, such as skin temperature, heart rate and skin conductance, without any change in the architecture. A neural attention module that we designed specifically for time series prediction improves prediction performance on synthetic data; however, the improvements do not transfer to real data. Conversely, using time of day as an additional input feature consistently improves the LSTM performance on real data but not on synthetic data. These and other differences show that behavior on synthetic data cannot be assumed to always transfer to real data, highlighting the importance of evaluating physiological models on data from real patients.