Natural rates are an important concept within the new Keynesian models often used for monetary policy advice. However, many of these models rely on demeaned interest rate and inflation data. Thus, they implicitly impose the strict assumption that the natural rates of these series are constant. Using New Zealand data and a small open-economy new Keynesian model with time-varying parameters, we estimate the natural real rate of interest, inflation target, potential output, and neutral real exchange rate. We find that the model estimates of the natural real rate of interest and neutral exchange rate display noticeable time variation and considerable uncertainty, while the inflation target has been relatively stable over the sample period. We also compare the results of this model to a model with time-invariant natural rates. The comparison reveals the data prefers the fit of the timevarying model. It also shows that allowing the natural rates to vary over time has implications for the persistence parameters and impulse responses of the model.