Skip to content

Fix sample interval calculation on DST transitions and kimber on tz-naive data - #2937

Open
pranaysb wants to merge 1 commit into
pvlib:mainfrom
pranaysb:fix-dst-naive-intervals
Open

pranaysb wants to merge 1 commit into
pvlib:mainfrom
pranaysb:fix-dst-naive-intervals

Conversation

@pranaysb

@pranaysb pranaysb commented Nov 8, 2026

Copy link
Copy Markdown

Description

Fixes two issues identified in #2935 following the pandas 3.1.0 compatibility updates:

  1. detect_clearsky & temperature.prilliman on DST transition days: In pvlib.tools._get_sample_intervals, consecutive sample intervals were previously computed using times.tz_localize(None).to_numpy(). Stripping timezone metadata converted timestamps to local wall-clock time, causing spring-forward daylight saving transitions to register as a 61-minute step and triggering NotImplementedError: algorithm does not yet support unequal time intervals. Consecutive intervals are now calculated using direct DatetimeIndex subtraction (times[1:] - times[:-1]) / pd.Timedelta('60s'), which correctly preserves elapsed physical duration for timezone-aware series across DST transitions while supporting timezone-naive indices.
  2. soiling.kimber on timezone-naive data: In pvlib.soiling.kimber, calling rainfall.index.tz_convert(None) failed on timezone-naive inputs with TypeError: Cannot convert tz-naive timestamps. Timestep intervals and day fractions are now calculated directly from rainfall.index[1] - rainfall.index[0] / pd.Timedelta(days=1), supporting both timezone-naive and timezone-aware indexes.

Testing

  • Added regression tests in tests/test_clearsky.py, tests/test_temperature.py, tests/test_soiling.py, and tests/test_tools.py.
  • Verified all 1463 unit tests pass with pytest.
  • Verified style conforms with flake8.

@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown

Hey @pranaysb! 🎉

Thanks for opening your first pull request! We appreciate your
contribution. Please ensure you have reviewed and understood the
contributing guidelines.

If AI is used for any portion of this PR, you must vet the content
for technical accuracy.

Finally, be sure the PR description includes the PR
checklist,
and complete the items you are able to.

@o1-shevchenko

Copy link
Copy Markdown

Would a fall-back regression be useful alongside the spring-forward test? For example:

times = pd.date_range(
    "2026-11-01 05:00", periods=8, freq="15min", tz="UTC"
).tz_convert("America/New_York")
assert tools._get_sample_intervals(times, 60) == (15.0, 4)

I checked the timestamps independently: 01:00–01:45 occurs twice with different UTC offsets, while every elapsed interval is 15 minutes. That makes the case useful for handover exports containing a repeated local hour. The assertion above is a proposed regression, not a result from running this PR.

…aive data

Fixes pvlib#2935

In pvlib.tools._get_sample_intervals, consecutive timestamp deltas are now
computed using direct DatetimeIndex subtraction rather than stripping
timezones via tz_localize(None), preventing spurious 61-minute interval steps
during Daylight Saving Time (DST) transitions.

In pvlib.soiling.kimber, timestep intervals and day fractions are computed
directly from index timedelta without calling tz_convert(None), preventing
TypeError when given timezone-naive Series.

Added regression tests in test_clearsky, test_temperature, test_soiling,
and test_tools.
@pranaysb
pranaysb force-pushed the fix-dst-naive-intervals branch from 33566fe to a4fd451 Compare November 15, 2026 17:47
@pranaysb

Copy link
Copy Markdown
Author

Thanks @o1-shevchenko, I've added the fall-back regression test to tests/test_tools.py alongside the spring-forward case and updated the branch.

@wholmgren wholmgren left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pranaysb please address the comments below and complete the original PR checklist including the AI attribution line.

Comment thread tests/test_clearsky.py
ghi = loc.get_clearsky(times)['ghi']
result = clearsky.detect_clearsky(ghi, ghi)
assert isinstance(result, pd.Series)
assert len(result) == len(times)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linked issue suggested the following which is not implemented here and seems like a better test than what you currently have: detect_clearsky on a DST day should match the same data in UTC

is scalar (:issue:`2881`, :pull:`2882`)
* :py:func:`pvlib.scaling.wvm` no longer returns all NaNs when ``positions``
contains a single site (:issue:`2895`, :pull:`2896`)
* Fix :py:func:`pvlib.clearsky.detect_clearsky` and :py:func:`pvlib.temperature.prilliman`

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

users never saw this bug so I don't think it belongs in the changelog. Instead add the issue reference to the pandas 3.1.0 enhancement line, as well as this PR number.

Comment thread tests/test_tools.py
assert count == 20


def test_get_sample_intervals_unequal():

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the function is currently implemented, this test exercises the pd.infer_freq route rather than the manual differences that you're touching in this PR. Probably still a good addition here.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would exercise the unequal delta path:

@pytest.mark.parametrize('times', [
    # months differ in length
    pd.date_range('2026-01-01', periods=4, freq='MS'),
    # local midnights: the day of the spring DST change is 23 h long
    pd.date_range('2026-03-06', periods=4, freq='D').tz_localize(
        'America/Denver'),
])
def test__get_sample_intervals_calendar_freq(times):
    # pandas finds a frequency for these, so only the check of the time
    # elapsed between samples can reject them
    assert tools._pandas_infer_freq_BaseOffset_transition_handler(times)
    with pytest.raises(NotImplementedError, match='unequal time intervals'):
        tools._get_sample_intervals(times, 20)

* :ghuser:`bfs2021`
* :ghuser:`xiajiahang`
* Jakub Krasuski (:ghuser:`Blizzeq`)
* S B Pranay (:ghuser:`pranaysb`)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add @aarmens702-hub as well

Comment thread pvlib/tools.py
np.diff(times.tz_localize(None).to_numpy()) # tz-naive np.datetime64
/ np.timedelta64(1, '60s')
)
if not isinstance(times, pd.DatetimeIndex):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this added? coercing to DatetimeIndex is scope creep for this function. If we need to keep it then it needs to be tested, and I don't think that's the case in this PR.

@wholmgren wholmgren added this to the v0.16.2 milestone Nov 18, 2026
@wholmgren wholmgren added the bug label Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

detect_clearsky and prilliman fail on DST days, kimber fails on tz-naive data (since #2924)

3 participants