Introduction

This spring I installed a photovoltaic (PV) system on my roof to generate renewable energy for my home. Soon after, I wrote a Home Assistant automation to optimize solar use. You can read more in my previous post Home Assistant solar energy management.

Summer, with plenty of sunshine, didn’t really challenge the YAML-based setup. But as autumn and winter approached — with shorter days, less predictable weather, and increasing heating energy consumption — I realized I needed something more robust than a collection of YAML automations.

It was the perfect chance to put lessons from Fluent Python into practice. 😂

After two to three months of tinkering in my spare time, I implemented a new solar and heating energy management system using AppDaemon. Here are some stats about the project:

• 📦 2,529 lines of production code across 55 Python files
• 🧪 4,248 lines of test code in 37 test files
• 🧾 476 test scenarios executed (170 test functions expanded through parametrization)
• ✅ 94% test coverage

I originally planned to describe every part of the project, but that would be too much for most readers. So I split the post into three parts:

1. Configuration snippets that reveal the system’s complexity without overwhelming.
2. A closer look at a few Python techniques I used, could be useful even if you aren’t interested in renewable energy.
3. Selected algorithms and how they work in practice.

AppDaemon applications

AppDaemon is a lightweight Python daemon I use alongside Home Assistant. It listens to events and instantiates small Python classes (apps) that register callbacks, read state, and call services. I installed AppDaemon as Home Assistant add-on so it runs in a docker container inside Home Assistant.

Solar

The first app is responsible for solar energy management:

• Setting up battery reserve SoC control every 5 minutes
• Setting up storage mode control triggers
• Setting up battery discharge schedule

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import appdaemon.plugins.hass.hassapi as hass

class SolarApp(hass.Hass):
    def initialize(self) -> None:
        self.solar = Solar(
            configuration=configuration,
            state_factory=state_factory,
            battery_discharge_slot_estimator=BatteryDischargeSlotEstimator(...),
            battery_reserve_soc_estimator=BatteryReserveSocEstimator(...),
            storage_mode_estimator=StorageModeEstimator(...),
        )

        self.run_every(self.control_battery_reserve_soc, "00:00:00", 5 * 60)
        self.listen_state(
            self.control_storage_mode, 
            [BATTERY_SOC_ENTITY, PRICE_FORECAST_ENTITY],
            constrain_start_time="sunrise +01:00:00",
            constrain_end_time="sunset -01:00:00",
        )
        self.run_daily(self.schedule_battery_discharge, "15:30:00")
        self.run_daily(self.disable_battery_discharge, "22:00:00")

    def control_battery_reserve_soc(self, **kwargs: object) -> None:
        self.solar.control_battery_reserve_soc(self.get_now())

    def control_storage_mode(self, entity, attribute, old, new, **kwargs) -> None:
        self.solar.control_storage_mode(self.get_now())

    def schedule_battery_discharge(self, **kwargs: object) -> None:
        self.solar.schedule_battery_discharge(self.get_now())

    def disable_battery_discharge(self, **kwargs: object) -> None:
        self.solar.disable_battery_discharge()

As you can see, the app mostly wires AppDaemon framework with the core Solar class that implements the actual logic. This is a design strategy because AppDaemon has limited support for testing.

The interesting part is the configuration of the Solar class to externalize all domain-specific parameters.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
configuration = SolarConfiguration(
    # nominal battery capacity
    battery_capacity=EnergyKwh(10.0),
    # nominal battery voltage
    battery_voltage=BatteryVoltage(52.0),
    # maximum battery discharge/charge current
    battery_maximum_current=BatteryCurrent(80.0),
    # minimum reserve SOC
    battery_reserve_soc_min=BatterySoc(20.0),
    # margin above minimum reserve SOC
    battery_reserve_soc_margin=BatterySoc(8.0),
    # upper limit when charging from the grid
    battery_reserve_soc_max=BatterySoc(90.0),
    # indoor temperature setpoint to estimate heating needs
    temp_in=Celsius(21.0),
    # outdoor temperature threshold to apply heating energy consumption in eco mode
    temp_out_threshold=Celsius(2.0),
    # coefficient of heat-pump performance at 7 degrees Celsius
    heating_cop_at_7c=4.0,
    # coefficient representing building heat loss rate in kW/°C
    heating_h=0.18,
    # outdoor temperature if weather forecast isn't available
    temp_out_fallback=Celsius(2.0),
    # outdoor humidity if weather forecast isn't available
    humidity_out_fallback=80.0,
    # regular consumption when in away mode
    regular_consumption_away=EnergyKwh(0.35),
    # consumption during daytime
    regular_consumption_day=EnergyKwh(0.5),
    # consumption during evening
    regular_consumption_evening=EnergyKwh(0.8),
    # threshold for exporting PV energy, net price
    pv_export_min_price_margin=EnergyPrice.pln_per_mwh(Decimal(200)),
    # threshold for exporting battery energy, net price
    battery_export_threshold_price=EnergyPrice.pln_per_mwh(Decimal(1000)),
    # skip battery export below this threshold
    battery_export_threshold_energy=EnergyKwh(1.0),
    # start time of night low tariff period (with margin)
    night_low_tariff_time_start=time.fromisoformat("22:05:00"),
    # end time of night low tariff period (with margin)
    night_low_tariff_time_end=time.fromisoformat("06:55:00"),
    # start time of day low tariff period (with margin)
    day_low_tariff_time_start=time.fromisoformat("13:05:00"),
    # end time of day low tariff period (with margin)
    day_low_tariff_time_end=time.fromisoformat("15:55:00"),
)

Static configuration is complemented by dynamic state information about the system taken from Home Assistant sensors and integrations:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
class SolarState:
    battery_soc: BatterySoc # current battery state of charge
    battery_reserve_soc: BatterySoc # current battery reserve state of charge
    is_away_mode: bool # away mode status
    is_eco_mode: bool # eco mode status
    inverter_storage_mode: StorageMode # current inverter storage mode
    is_slot1_discharge_enabled: bool # whether slot 1 discharge is enabled
    slot1_discharge_time: str # discharge time for slot 1
    slot1_discharge_current: BatteryCurrent # discharge current for slot 1
    hvac_heating_mode: str # heating mode
    hourly_price: EnergyPrice # current hourly energy price
    pv_forecast_today: list # today's PV forecast
    pv_forecast_tomorrow: list # tomorrow's PV forecast
    weather_forecast: dict | None # weather forecast data
    price_forecast: list | None # energy price forecast

Like a tip of the iceberg, configuration and state should give you an idea what the system takes into account when making autonomous decisions 🤔 The only manual input is the eco/away mode toggles.

Heating, cooling and domestic hot water

The second app manages my heat pump: domestic hot water and heating or cooling. Nothing too fancy, just periodic control and reaction to eco mode and heating/cooling mode changes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import appdaemon.plugins.hass.hassapi as hass

class HvacApp(hass.Hass):
    def initialize(self) -> None:
        self.hvac = Hvac(
            configuration=configuration,
            state_factory=state_factory,
            dhw_estimator=DhwEstimator(...),
            heating_estimator=HeatingEstimator(...),
            cooling_estimator=CoolingEstimator(...),
        )
        self.run_every(self.control_scheduled, "00:00:00", 5 * 60)

        self.listen_state(
            self.control_triggered,
            [ECO_MODE_ENTITY, HEATING_ENTITY,  COOLING_ENTITY],
        )

    def control_scheduled(self, **kwargs: dict) -> None:
        self.hvac.control(self.get_now())

    def control_triggered(self, entity, attribute, old, new, **kwargs) -> None:
        self.hvac.control(self.get_now())        

Again the most interesting part is the static configuration and dynamic state of the Hvac class. Everything is automated, the only manual input are the eco mode toggle and temperature adjustment (+/- 1 degree).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
configuration = HvacConfiguration(
    # domestic hot water temperature
    dhw_temp=Celsius(48.0),
    # domestic hot water temperature in eco mode
    dhw_temp_eco=Celsius(40.0),
    # when to start boosting DHW depends on temperature difference
    dhw_delta_temp=Celsius(6.0),
    # 5 minutes after low tariff starts to avoid clocks drift issues
    dhw_boost_start=time.fromisoformat("13:05:00"),
    # 5 minutes before high tariff starts to avoid clocks drift issues
    dhw_boost_end=time.fromisoformat("15:55:00"),
    # heating temperature
    heating_temp=Celsius(20.0),
    # heating temperature in eco mode
    heating_temp_eco=Celsius(18.0),
    # heating boost delta
    heating_boost_delta_temp=Celsius(1.0),
    # heating boost delta in eco mode
    heating_boost_delta_temp_eco=Celsius(2.0),
    # 5 minutes after low tariff starts to avoid clocks drift issues
    heating_boost_time_start_eco_on=time.fromisoformat("22:05:00"),
    # 15 minutes before high tariff starts because stop heating takes longer
    heating_boost_time_end_eco_on=time.fromisoformat("06:45:00"),
    # 1 hour before wake up time
    heating_boost_time_start_eco_off=time.fromisoformat("05:00:00"),
    # 1 hour before bed time
    heating_boost_time_end_eco_off=time.fromisoformat("21:00:00"),
    # cooling temperature
    cooling_temp=Celsius(24.0),
    # cooling temperature in eco mode
    cooling_temp_eco=Celsius(26.0),
    # cooling boost delta
    cooling_boost_delta_temp=Celsius(2.0),
    # cooling boost delta in eco mode
    cooling_boost_delta_temp_eco=Celsius(2.0),
    # cool when there is plenty of solar energy
    cooling_boost_time_start_eco_on=time.fromisoformat("12:00:00"),
    cooling_boost_time_end_eco_on=time.fromisoformat("16:00:00"),
    # extends cooling period a bit when eco mode is off
    cooling_boost_time_start_eco_off=time.fromisoformat("10:00:00"),
    cooling_boost_time_end_eco_off=time.fromisoformat("18:00:00"),
)

1
2
3
4
5
6
7
8
9
10
class HvacState:
    is_eco_mode: bool  # eco mode status
    dhw_actual_temperature: Celsius  # actual domestic hot water temperature
    dhw_target_temperature: Celsius  # target domestic hot water temperature
    indoor_actual_temperature: Celsius  # actual indoor temperature
    heating_target_temperature: Celsius  # target heating temperature
    heating_mode: str  # heating mode
    cooling_target_temperature: Celsius  # target cooling temperature
    cooling_mode: str  # cooling mode
    temperature_adjustment: Celsius  # temperature adjustment, +-1 degree

If you’d like to see the complete implementation, explore the source on GitHub: https://github.com/mkuthan/home-assistant-appdaemon. The main branch contains the production code I run at home.

Python goodies

Let’s move to the second part of the post, where I describe some interesting Python techniques I applied in the project.

Typing

As a seasoned Java and Scala developer, I appreciate strong typing. In Python typing is optional, but I decided to use it extensively to check it’s maturity. I configured my build and CI pipeline to use pyright from Microsoft and ty from Astral for type checking.

“Pyright” has the advantage of being fully compatible with Pylance in VS Code. However, I found “ty” to be blazingly fast — like other Astral tools such as ruff. The Astral tool is the clear winner in this comparison.

1
2
3
4
5
6
7
$ time pyright
Found 95 source files
0 errors, 0 warnings, 0 informations

real    0m2.661s
user    0m3.672s
sys     0m0.259s

1
2
3
4
5
6
7
$ time ty check
INFO Indexed 95 file(s) in 0.003s
All checks passed!

real    0m0.125s
user    0m0.250s
sys     0m0.059s

Value classes

Around 2013 I read Domain-Driven Design: Tackling Complexity in the Heart of Software by Eric Evans. One of the key takeaways was to use value objects to represent domain concepts instead of bunch of primitive types, like float for energy, temperature, etc. I implemented value classes in Python using the @dataclass(frozen=True) decorator and a set of dunder (magic) methods for seamless integration with the Python SDK. For example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
@dataclass(frozen=True, order=True)
class EnergyKwh:
    _ZERO_VALUE: ClassVar[float] = 0.0

    value: float

    def __add__(self, other: "EnergyKwh") -> "EnergyKwh":
        return EnergyKwh(value=self.value + other.value)

    def __sub__(self, other: "EnergyKwh") -> "EnergyKwh":
        return EnergyKwh(value=self.value - other.value)

    def __truediv__(self, other: "EnergyKwh") -> float:
        if other == ENERGY_KWH_ZERO:
            raise ValueError("Cannot divide by zero energy")
        return self.value / other.value

    def __neg__(self) -> "EnergyKwh":
        return EnergyKwh(value=-self.value)

    def __str__(self) -> str:
        return f"{self.value:.2f}kWh"


ENERGY_KWH_ZERO = EnergyKwh(EnergyKwh._ZERO_VALUE)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
class Celsius:
    _ZERO_VALUE: ClassVar[float] = 0.0

    value: float

    def __add__(self, other: "Celsius") -> "Celsius":
        return Celsius(value=self.value + other.value)

    def __sub__(self, other: "Celsius") -> "Celsius":
        return Celsius(value=self.value - other.value)

    def __mul__(self, other: float) -> "Celsius":
        return Celsius(value=self.value * other)

    def __truediv__(self, other: "Celsius") -> float:
        if other == CELSIUS_ZERO:
            raise ValueError("Cannot divide by zero temperature")
        return self.value / other.value

    def __round__(self) -> "Celsius":
        return Celsius(value=floor(self.value + 0.5))

    def __str__(self) -> str:
        return f"{self.value:.1f}°C"


CELSIUS_ZERO = Celsius(Celsius._ZERO_VALUE)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
@dataclass(frozen=True, order=True)
class BatterySoc:
    _MIN_VALUE: ClassVar[float] = 0.0
    _MAX_VALUE: ClassVar[float] = 100.0

    value: float

    def __post_init__(self) -> None:
        if not self._MIN_VALUE <= self.value <= self._MAX_VALUE:
            raise ValueError(f"Battery SOC must be between {self._MIN_VALUE} and {self._MAX_VALUE}, got {self.value}")

    def __add__(self, other: "BatterySoc") -> "BatterySoc":
        return BatterySoc(value=min(self.value + other.value, self._MAX_VALUE))

    def __sub__(self, other: "BatterySoc") -> "BatterySoc":
        return BatterySoc(value=max(self.value - other.value, self._MIN_VALUE))

    def __round__(self) -> "BatterySoc":
        return BatterySoc(value=floor(self.value + 0.5))

    def __str__(self) -> str:
        return f"{self.value:.2f}%"


BATTERY_SOC_MIN = BatterySoc(BatterySoc._MIN_VALUE)
BATTERY_SOC_MAX = BatterySoc(BatterySoc._MAX_VALUE)

With value classes I gained several benefits:

• With dataclass you get boilerplate code generation for free (__init__, __eq__, etc.)
• With frozen dataclass you get immutability for free
• With order=True you get comparison operators for free
• The type checker warns if you try to add an EnergyKwh value to a Celsius temperature
• You can’t create invalid values — for example, BatterySoc is constrained to the 0–100% range
• You get consistent string representation across the application
• You can add domain-specific methods to value classes, e.g., rounding temperature values

Protocols

AppDeamon support for testing is limited, so I had to extract logic from the framework-specific code and make it testable in isolation. I decided to use duck typing with Protocol classes to define AppDaemon interfaces, for example:

1
2
3
4
5
6
7
8
class AppdaemonService(Protocol):
    def call_service(self, service: str, **data) -> object: ...
    
class AppdaemonState(Protocol):
    def get_state(self, entity_id: str, attribute: str | None = None) -> object: ...

class AppdaemonLogger(Protocol):
    def log(self, msg: str, *args, level: str | int = logging.INFO) -> None: ...

Those three protocols are all I needed to interact with AppDaemon framework in my applications! The protocols initialization is straightforward because the protocols match the AppDaemon API:

1
2
3
4
5
6
7
8
import appdaemon.plugins.hass.hassapi as hass

class HvacApp(hass.Hass):
    def initialize(self) -> None:
        appdaemon_logger = self
        appdaemon_state = self
        appdaemon_service = self
        (...)

For testing I define mock implementations of those protocols in the top level conftest.py file. The fixtures are automatically available in all test modules without the need to monkey-patch AppDaemon classes.

1
2
3
4
5
6
7
8
9
10
11
12
13
@pytest.fixture
def mock_appdaemon_logger() -> Mock:
    return Mock()


@pytest.fixture
def mock_appdaemon_state() -> Mock:
    return Mock()


@pytest.fixture
def mock_appdaemon_service() -> Mock:
    return Mock()

For comprehensions

I love Scala’s for-comprehensions, and I also enjoy their Python equivalent. For example, I implemented separate classes for different consumption-forecast strategies and combined them with a composite class. Using a nested for clause inside a single list comprehension is an elegant way to implement the composite design pattern.

1
2
3
4
5
6
7
8
9
class ConsumptionForecast(Protocol):
    def hourly(self, period_start: datetime, period_hours: int) -> list[HourlyConsumptionEnergy]: ...

class ConsumptionForecastComposite:
    def __init__(self, *components: ConsumptionForecast) -> None:
        self.components = components

    def hourly(self, period_start: datetime, period_hours: int) -> list[HourlyConsumptionEnergy]:
        return [item for component in self.components for item in component.hourly(period_start, period_hours)]

Interesting algorithms

Algorithms have never been my top strength but with help from GitHub Copilot and a basic grasp of intuition, math and physics, I implemented a few useful ones.

Find the continuous time window with maximum revenue

This algorithm evaluates all possible starting minutes to find the optimal battery discharge window with maximum revenue. It uses a variable-length sliding window approach with time complexity O(n * d) where “n” is the number of periods and “d” is max_duration_minutes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
def find_max_revenue_period(
    hourly_prices: list[HourlyPrice], min_price_threshold: EnergyPrice, max_duration_minutes: int
) -> tuple[EnergyPrice, datetime, datetime] | None:
    if max_duration_minutes < 1:
        raise ValueError(f"max_duration_minutes must be at least 1, got {max_duration_minutes}")

    if not hourly_prices:
        return None

    if not any(hourly_price.price >= min_price_threshold for hourly_price in hourly_prices):
        return None

    period_duration_minutes = 60

    max_revenue = None
    best_start_time = None
    best_end_time = None

    for start_hour_idx, start_hour in enumerate(hourly_prices):
        # Skip if period doesn't meet price threshold
        if start_hour.price < min_price_threshold:
            continue

        for start_offset_minutes in range(period_duration_minutes):
            start_time = start_hour.period.start + timedelta(minutes=start_offset_minutes)

            # Calculate revenue for a window of max_duration_minutes from this start
            revenue = start_hour.price.zeroed()
            minutes_covered = 0

            # Iterate through periods that this window spans
            current_period_idx = start_hour_idx
            minutes_into_current_period = start_offset_minutes

            while minutes_covered < max_duration_minutes and current_period_idx < len(hourly_prices):
                current_hourly_price = hourly_prices[current_period_idx]

                # Check threshold - stop if this period doesn't meet it
                if current_hourly_price.price < min_price_threshold:
                    break

                # Calculate how many minutes to take from this period
                minutes_available_in_period = period_duration_minutes - minutes_into_current_period
                minutes_needed = max_duration_minutes - minutes_covered
                minutes_to_take = min(minutes_available_in_period, minutes_needed)

                # Add revenue
                price_per_minute = current_hourly_price.price / Decimal(period_duration_minutes)
                revenue += price_per_minute * Decimal(minutes_to_take)
                minutes_covered += minutes_to_take

                # Move to next period
                current_period_idx += 1
                minutes_into_current_period = 0

            # Check if this is a valid and better solution
            if minutes_covered >= 1:
                end_time = start_time + timedelta(minutes=minutes_covered)

                if max_revenue is None or revenue > max_revenue:
                    max_revenue = revenue
                    best_start_time = start_time
                    best_end_time = end_time

    if max_revenue is None or best_start_time is None or best_end_time is None:
        return None

    return (max_revenue, best_start_time, best_end_time)

For the following hourly prices:

1
2
3
4
5
6
[
    ("00:00:00", 100),
    ("01:00:00", 150),
    ("02:00:00", 200),
    ("03:00:00", 120),
]

The maximum revenue for a 105 minutes window with minimum price threshold of “100” is 150 * 45 / 60 + 200 = 312.5, starting at “01:15:00” and ending at “03:00:00”.

Estimate heating energy consumption

This function estimates the electrical energy consumption of a heat pump by accounting for:

• Heat loss proportional to indoor/outdoor temperature difference
• COP (Coefficient of Performance) variation with outdoor temperature
• COP degradation due to frosting cycles in specific temperature and humidity ranges

The model assumes linear heat loss and uses empirically-derived adjustments for real-world heat pump performance characteristics.

For example, for the outdoor temperature of 3.5°C, indoor temperature of 20°C, humidity of 100% with frosting penalty, COP at 7°C of 4.0, and heat loss coefficient of 0.18 kW/°C, the estimated heating energy consumption is approximately 0.987 kWh.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
_REFERENCE_TEMPERATURE = Celsius(7.0)  # temperature at which COP is rated
_COP_TEMPERATURE_COEFFICIENT = 0.033  # COP change per degree Celsius
_COP_COEFFICIENT_MIN = 0.5  # minimum COP multiplier to prevent unrealistic values

# Frosting cycle parameters
_FROSTING_TEMP_MIN = Celsius(0.0)  # lower bound for frosting conditions
_FROSTING_TEMP_MAX = Celsius(7.0)  # upper bound for frosting conditions
_FROSTING_TEMP_PEAK = Celsius(3.5)  # temperature with maximum frosting risk
_FROSTING_HUMIDITY_THRESHOLD = 70.0  # % - minimum humidity for frosting
_FROSTING_PENALTY_MAX = 0.15  # maximum COP reduction due to frosting cycles (15%)


def estimate_heating_energy_consumption(
    t_out: Celsius,
    t_in: Celsius,
    humidity: float,
    cop_at_7c: float,
    h: float,
) -> EnergyKwh:
    t_diff = t_in - t_out

    if t_diff.value > 0:
        heat_loss = h * t_diff.value

        temperature_coefficient = _temperature_coefficient(t_out)
        frosting_penalty = _frosting_penalty(t_out, humidity)
        adjusted_cop = cop_at_7c * temperature_coefficient * frosting_penalty

        energy_consumption = EnergyKwh(heat_loss / adjusted_cop)
    else:
        energy_consumption = ENERGY_KWH_ZERO

    return energy_consumption


def _temperature_coefficient(
    t_out: Celsius,
) -> float:
    temp_delta = t_out - _REFERENCE_TEMPERATURE
    coefficient = 1.0 + (temp_delta.value * _COP_TEMPERATURE_COEFFICIENT)

    return max(_COP_COEFFICIENT_MIN, coefficient)


def _frosting_penalty(
    t_out: Celsius,
    humidity: float,
) -> float:
    if not (_FROSTING_TEMP_MIN <= t_out <= _FROSTING_TEMP_MAX and humidity > _FROSTING_HUMIDITY_THRESHOLD):
        return 1.0

    distance_from_peak = abs(t_out.value - _FROSTING_TEMP_PEAK.value)
    frosting_severity = (_FROSTING_TEMP_PEAK.value - distance_from_peak) / _FROSTING_TEMP_PEAK.value

    humidity_factor = (humidity - _FROSTING_HUMIDITY_THRESHOLD) / (100.0 - _FROSTING_HUMIDITY_THRESHOLD)

    penalty = _FROSTING_PENALTY_MAX * frosting_severity * humidity_factor

    return 1.0 - penalty

Calculate maximum cumulative energy deficit

To calculate the battery reserve SOC in the morning, I need to find the maximum cumulative energy deficit before next low tariff period. For example, to survive this hypothetical morning the battery needs to cover 1.25 kWh deficit:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
def maximum_cumulative_deficit(
    consumptions: list[HourlyConsumptionEnergy], productions: list[HourlyProductionEnergy]
) -> EnergyKwh:
    net_energy_dict = {}

    for consumption in consumptions:
        if consumption.period in net_energy_dict:
            net_energy_dict[consumption.period] -= consumption.energy
        else:
            net_energy_dict[consumption.period] = -consumption.energy

    for production in productions:
        if production.period in net_energy_dict:
            net_energy_dict[production.period] += production.energy
        else:
            net_energy_dict[production.period] = production.energy

    net_energy_list = list(net_energy_dict.items())
    net_energy_list_sorted = sorted(net_energy_list, key=lambda n: n[0].start)

    cumulative_balance = ENERGY_KWH_ZERO
    min_cumulative_balance = ENERGY_KWH_ZERO

    for net_energy in net_energy_list_sorted:
        cumulative_balance = cumulative_balance + net_energy[1]

        if cumulative_balance < min_cumulative_balance:
            min_cumulative_balance = cumulative_balance

    return max(-min_cumulative_balance, ENERGY_KWH_ZERO)

Estimate time for indoor temperature to decay from start to end temperature

This function estimates the time required for indoor temperature to decay from a starting temperature to an ending temperature, considering varying outdoor temperatures over time. It uses Newton’s Law of Cooling with a decay rate constant derived from the building’s thermal properties.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
def estimate_temperature_decay_time(
    temp_start: Celsius,
    temp_end: Celsius,
    hourly_weather: list[HourlyWeather],
    decay_rate: float,
) -> timedelta:
    if decay_rate <= 0:
        raise ValueError("Decay rate must be positive")

    temp_current = temp_start
    temp_target = temp_end

    total_hours = 0.0

    for weather in hourly_weather:
        temp_outdoor = weather.temperature

        if temp_current <= temp_target:
            break

        if temp_current <= temp_outdoor:
            break

        temp_diff_start = temp_current - temp_outdoor
        temp_after_hour = temp_outdoor + Celsius(temp_diff_start.value * exp(-decay_rate * 1.0))

        if temp_after_hour <= temp_target:
            fraction = log(temp_diff_start / (temp_target - temp_outdoor)) / decay_rate
            total_hours += fraction
            break

        temp_current = temp_after_hour
        total_hours += 1.0

    return timedelta(hours=total_hours)

For example, starting from 22.9°C to 21.3°C with constant outdoor temperature of 8.0°C and decay rate of 0.0226, the estimated time is approximately 5.03 hours.

Summary

I had a lot of fun building this project and learning Python along the way. During implementation I deliberately avoided some language features from the book like: multiple inheritance and generics, because I’m not a big fan. I still plan to explore coroutines, async/await, and metaclasses in more depth.

Getting back to the point, since the initial deployment in early autumn:

• The house stays warm and cozy.
• My family hasn’t complained about hot water.
• Most grid consumption now happens in low‑tariff periods: 90% of grid energy is used then (total cost 0.58 PLN/kWh), while only 10% is used during high‑tariff periods (total cost 1.06 PLN/kWh).
• Self‑consumption of solar energy was 59% in October and 70% in November.

Don’t forget to add a ⭐️ to my project on GitHub if you find it useful! https://github.com/mkuthan/home-assistant-appdaemon

After some deep research, we discovered that TechniSchools is a perfect match for him. My son decided to apply next year, but first he needs to do some serious homework in programming and other technical subjects. He already knows Scratch and a bit of C (Arduino), Python, and MicroPython (Lego Mindstorms), but I suggested he should learn web technologies as well (HTML, CSS, JavaScript).

He started with W3Schools video tutorials, switched to VS Code with some Copilot help, and created a few simple web pages. It was his way of learning by doing—typical for his generation, I assume.

But recently he asked me to recommend some books 📚 that would help him understand programming concepts better. I was super happy 😊 because I know that reading books is the best way to learn programming fundamentals deeply.

I started searching for books, but most of them were… just boring 😴 (at least for a teenager). Then I remembered the Head First series from O’Reilly.

• Head First Python by Paul Barry
• Head First JavaScript Programming by Eric Freeman and Elisabeth Robson

My son started with Head First JavaScript and I started leafing through Head First Python. First, you read about learning psychology and why these books are a bit different from typical programming books: lots of pictures, funny stories, quizzes, exercises, etc. Then you dive into the project-based learning approach, building simple applications step by step. In the JavaScript book, you create a ship battle game; in the Python book, you build a web app for swimming club training recordings.

I’m really impressed by how cleverly the books smuggle in programming concepts and best practices I usually apply in my daily work as a software engineer. For example: make small iterations, prefer readability over cleverness, or apply the YAGNI principle.

Dear parents, I highly recommend these books for your kids interested in programming!

PV installation

I have a 6.3 kWp PV system with a 10 kWh battery storage unit installed in my utility room.

At the moment I am buying electricity at a flat rate of 1.2 PLN/kWh (0.28 €/kWh) including all taxes and fees. I can also sell excess energy back to the grid at dynamic hourly rates. The selling price depends on the time of the day and the current demand for energy. Charts below show the hourly rates for today and tomorrow, in PLN/MWh (to get the price in EUR or USD, just divide by ~4.3).

Requirements

To make the most of my solar energy system, I have outlined the following key requirements for efficient energy management and automation:

⚡️ Prioritize self-consumption over exporting energy to the grid, as it’s the most cost-effective approach for my setup.

🌍 Export energy to the grid only when the current and forecasted household consumption is fully covered by on-site generation.

🚫 Limit energy export to the grid when hourly electricity prices are zero or negative. Exporting under such conditions provides no financial benefit and only generates unnecessary heat in the inverter.

Inverter remote control

The most important part of the setup is the ability to control the inverter remotely. I have a Solis hybrid inverter, which is compatible with the Solis Cloud API. Unfortunately, the existing solis-sensor integration covers only the monitoring part, the control part is experimental and unstable, see #437.

This project also provided an excellent opportunity to develop my own custom integration for Home Assistant, tailored specifically for controlling the Solis inverter. You can find the source code and detailed documentation here:

Solis Cloud Control Integration

At the time of writing, the integration allows for controlling the inverter in the following ways:

Energy Production Forecasting

An essential component of my setup is the ability to accurately forecast energy production. By integrating Solcast with Home Assistant, I can access detailed energy production forecasts tailored to my PV installation.

This integration provides valuable insights into expected solar generation, enabling better planning and optimization of energy usage and storage.

Solis Inverter Modes of Operation

Solis inverters support three primary modes of operation:

🔋 Self-Use: In this mode, the inverter prioritizes using energy generated by the PV system to power the home and charge the battery. Any excess energy is exported to the grid.

⚡ In-Feed Priority: This mode prioritizes selling energy to the grid. The battery will neither charge nor discharge unless “Time Charging” is enabled and properly configured.

🌐 Off-Grid: Designed for installations without grid power, this mode isn’t relevant to this project.

Automation for inverter mode

By default, my installation operates in Self-Use mode, which effectively handles most scenarios. However, under specific conditions, automation switches the inverter to In-Feed Priority mode to maximize energy export. These conditions include:

• The house is in “away mode,” indicating no one is home and excess energy is available.
• The sun is above the horizon, ensuring sufficient energy production.
• Hourly electricity prices are favorable and exceed the minimum price of the day.
• The remaining energy production forecast for the day is sufficient to fully recharge the battery, ensuring optimal energy storage for later use.
• Today’s temperature is high enough to minimize excessive power consumption for heating, considering the current state of charge (SOC) of the battery.
• The current household power consumption is low, which aligns with the current battery SOC to optimize energy usage.

In all other scenarios, the inverter reverts to Self-Use mode to optimize energy usage and storage.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
- alias: Solar - mode optimization
  id: solar_01
  triggers:
    - trigger: state
      entity_id: input_boolean.away_mode
    - trigger: time_pattern
      minutes: 0
      seconds: 30 # to get up-to-date hourly statistics
    - trigger: time_pattern
      minutes: 30
  variables:
    # Battery SOC [%]
    battery_soc: "{{ states('sensor.solis_remaining_battery_capacity') | float }}"
    battery_reserve: "{{ states('sensor.inverter_control_battery_reserve_soc') | float }}"
    battery_max_charge: "{{ states('sensor.inverter_control_battery_max_charge_soc') | float }}"
    battery_threshold_low: "{{ [battery_reserve | float + 10, battery_max_charge] | min }}"
    battery_threshold_high: "{{ [battery_reserve | float + 40, battery_max_charge] | min }}"

    # Prices [PLN/kWh]
    price: "{{ states('sensor.solar_electricity_price_hourly_rate') | float }}"
    price_min: "{{ states('sensor.solar_electricity_price_hourly_rate_min') | float }}"
    price_valley_threshold: "{{ states('input_number.solar_electricity_price_valley_threshold') | float }}"
    price_threshold: "{{ [price_min, price_valley_threshold] | max }}"

    # PV forecast [kWh]
    pv_forecast: "{{ states('sensor.solcast_pv_forecast_forecast_remaining_today') | float }}"
    pv_forecast_threshold_low: 15
    pv_forecast_threshold_high: 25

    # Temperature forecast [°C]
    temp_forecast: "{{ states('sensor.weather_temperature_today') | float }}"
    temp_forecast_threshold: 5

    # Power consumption [W]
    power_consumption: "{{ states('sensor.solar_load_power') | float }}"
    power_consumption_threshold: 500

    # Export power [W]
    export_power_capped: 1000
    export_power_nominal: 13200
    export_power: "{{ export_power_nominal if price > 0.01 else export_power_capped }}"

  actions:
    choose:
      - conditions:
          - condition: state
            entity_id: input_boolean.away_mode
            state: "on"
          - condition: state
            entity_id: sun.sun
            state: above_horizon
          - condition: template
            alias: If price is decent
            value_template: "{{ price > price_threshold }}"
          - condition: template
            alias: If PV forecast meets battery SOC criteria
            value_template: >
              {{ (pv_forecast > pv_forecast_threshold_high and battery_soc > battery_threshold_low) or 
                  (pv_forecast > pv_forecast_threshold_low and battery_soc > battery_threshold_high) }}
          - condition: template
            alias: If temperature meets battery SOC criteria
            value_template: >
              {{ (temp_forecast > temp_forecast_threshold and battery_soc > battery_threshold_low) or
                  (temp_forecast <= temp_forecast_threshold and battery_soc > battery_threshold_high) }}
          - condition: template
            alias: If power consumption meets battery SOC criteria
            value_template:
              "{{ (power_consumption < power_consumption_threshold and battery_soc > battery_threshold_low) or
              (power_consumption >= power_consumption_threshold and battery_soc > battery_threshold_high) }}"
        sequence:
          - action: select.select_option
            entity_id: select.inverter_control_storage_mode
            data:
              option: Feed-In Priority
          - action: number.set_value
            entity_id: number.inverter_control_max_export_power
            data:
              value: "{{ export_power }}"
      default:
        - action: select.select_option
          entity_id: select.inverter_control_storage_mode
          data:
            option: Self-Use
        - action: number.set_value
          entity_id: number.inverter_control_max_export_power
          data:
            value: "{{ export_power }}"

Selling energy during peak hour

The second automation schedules the inverter to sell energy during peak hour, when the following conditions are met:

• The house is working in eco mode. I have excess energy only when there is no one at home, or we’re going to leave soon.
• The price is decent and maximum of the day.
• Energy production forecast for tomorrow is excellent.
• Temperature forecast for tomorrow is high enough to minimize excessive power consumption for heating.
• Battery is fully charged.
• There will be enough energy for sale considering typical nighttime consumption.

The most tricky part of the automation is to calculate the amount of energy that can be sold. Ask LLM if you need more details about the algorithm 😜

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
- alias: Solar - schedule discharge slot
  id: solar_02
  triggers:
    - trigger: state
      entity_id: input_boolean.eco_mode
    - trigger: time
      at: "15:30:00" # when prices for the next day are available
    - trigger: time
      at: "16:00:00" # backup call
  variables:
    # Battery SOC [%]
    battery_soc: "{{ states('sensor.solis_remaining_battery_capacity') | float }}"
    battery_reserve: "{{ states('sensor.inverter_control_battery_reserve_soc') | float }}"
    battery_max_charge: "{{ states('sensor.inverter_control_battery_max_charge_soc') | float }}"
    battery_threshold_low: "{{ [battery_reserve | float + 15, battery_max_charge] | min }}"
    battery_threshold_high: "{{ [battery_max_charge | float - 10, battery_threshold_low] | max }}"

    battery_capacity: 10000 # Wh
    battery_voltage: "{{ states('sensor.solis_battery_voltage') | float }}" # V

    # PV forecast [kWh]
    pv_forecast: "{{ states('sensor.solcast_pv_forecast_forecast_tomorrow') | float }}"
    pv_forecast_threshold: 25

    # Temperature forecast [°C]
    temp_forecast: "{{ states('sensor.weather_temperature_tomorrow') | float }}"
    temp_forecast_threshold: 5

    # Peak price and time
    peak_raw: "{{ states('sensor.solar_electricity_price_hourly_rate_max_next_raw') | from_json }}"

    peak_price: "{{ peak_raw.price }}"
    peak_price_threshold: >
      {{ states('input_number.solar_electricity_price_peak_threshold') | float }}

    peak_time_hours: 1
    peak_time: >
      {% set from = '%02d:00' | format(peak_raw.hour) %}
      {% set to = '%02d:00' | format((peak_raw.hour + peak_time_hours) % 24) %}
      {{ from ~ "-" ~ to }}

    # Export energy [#Wh]
    energy_to_export_threshold: 1000
    energy_to_export: >
      {% set energy_available = (battery_soc - battery_threshold_low) / 100 * battery_capacity %}

      {% set sunset = as_timestamp(state_attr('sun.sun', 'next_setting')) %}
      {% set sunrise = as_timestamp(state_attr('sun.sun', 'next_rising')) %}
      {% if sunset > sunrise %}
          {% set sunset = sunset - 86400 %}
      {% endif %}
      {% set night_duration = sunrise - sunset  %}

      {% set night_avg_consumption = 300 %}
      {% set energy_night_consumption = (night_duration / 3600) * night_avg_consumption %}

      {{ [energy_available - energy_night_consumption, 0] | max }}

    # Export current [A]
    export_current_max: 100
    export_current: "{{ [energy_to_export / battery_voltage / peak_time_hours, export_current_max] | min }}"

  actions:
    choose:
      - conditions:
          - condition: state
            entity_id: input_boolean.eco_mode
            state: "on"
          - condition: template
            alias: If price is decent
            value_template: "{{ peak_price > peak_price_threshold }}"
          - condition: template
            alias: If PV forecast is good
            value_template: "{{ pv_forecast > pv_forecast_threshold }}"
          - condition: template
            alias: If temperature forecast is good
            value_template: "{{ temp_forecast > temp_forecast_threshold }}"
          - condition: template
            alias: If battery SOC is high
            value_template: "{{ battery_soc > battery_threshold_high }}"
          - condition: template
            alias: If there is enough energy to export
            value_template: "{{ energy_to_export > energy_to_export_threshold }}"
        sequence:
          - action: text.set_value
            entity_id: text.inverter_control_slot1_discharge_time
            data:
              value: "{{ peak_time }}"
          - action: number.set_value
            entity_id: number.inverter_control_slot1_discharge_current
            data:
              value: "{{ export_current }}"
          - action: switch.turn_on
            entity_id: switch.inverter_control_slot1_discharge
      default:
        - action: switch.turn_off
          entity_id: switch.inverter_control_slot1_discharge

Expose excess energy mode

The third automation introduces an “excess energy” mode, which can be utilized by other automations to determine when surplus energy is available for consumption. For instance, a bathroom heater automation can leverage this mode to activate the electric heater only when excess energy is detected.

Excess power sensor is calculated as follows:

1
2
3
4
5
6
7
8
9
10
11
12
template:
  - sensor:
      - name: Solar Excess Power
        unit_of_measurement: W
        device_class: power
        icon: mdi:lightning-bolt-outline
        state: >
          {% set pv = states('sensor.solar_pv_power') | float %}
          {% set battery = states('sensor.solar_battery_power') | float %}
          {% set load = states('sensor.solar_load_power') | float %}
          {% set excess = pv - (load + battery) if battery >= 0 else 0 %}
          {{ [excess, 0] | max }}

Automation for setting the excess energy mode is triggered by the following conditions:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
- alias: Solar - set excess energy mode "on"
  id: solar_03
  triggers:
    - trigger: numeric_state
      entity_id: sensor.solar_electricity_price_hourly_rate
      below: input_number.solar_electricity_price_valley_threshold
    - trigger: numeric_state
      entity_id: sensor.solar_excess_power
      above: 500
      for:
        minutes: 10
  conditions:
    - condition: numeric_state
      entity_id: sensor.solar_electricity_price_hourly_rate
      below: input_number.solar_electricity_price_valley_threshold
  actions:
    - action: input_boolean.turn_on
      entity_id: input_boolean.solar_excess_energy_mode

- alias: Solar - set excess energy mode "off"
  id: solar_04
  triggers:
    - trigger: numeric_state
      entity_id: sensor.solar_electricity_price_hourly_rate
      above: input_number.solar_electricity_price_valley_threshold
    - trigger: numeric_state
      entity_id: sensor.solar_excess_power
      below: 100
      for:
        minutes: 10
  conditions:
    - condition: numeric_state
      entity_id: sensor.solar_electricity_price_hourly_rate
      above: input_number.solar_electricity_price_valley_threshold
  actions:
    - action: input_boolean.turn_off
      entity_id: input_boolean.solar_excess_energy_mode

Summary

With a PV system of this size, I could have simply left the inverter in “Self-Use” mode, and the overall results wouldn’t differ significantly. However, I approached this entire experiment as an excellent opportunity to learn and have fun. By diving into the intricacies of energy management and automation, I gained valuable insights and hands-on experience that not only enhanced my technical skills but also made my home smarter and more efficient.

Don’t forget to add a ⭐️ to my project on GitHub if you find it useful! https://github.com/mkuthan/solis-cloud-control

Check it out here: https://github.com/mkuthan/homelab-public

Why IaaC?

What is a main challenge in a homelab? For me it’s the same as in a production environment - keeping everything up to date, secure, and reliable while minimizing manual work.

I’m a software engineer, so I’m used to writing beautiful, testable code. In the infrastructure world, it’s not that easy. Fortunately, IaaC tools like Terraform and Ansible help me to write infrastructure code in a way I’m used to.

Terraform

Terraform defines the following resources in my homelab:

🖥️ Linux containers (LXC) using the Telmate Proxmox provider. It covers most of the container resource definitions: CPU, memory, root disk, mount points, networking, SSH keys, and nested virtualization. In the Proxmox UI, I only define replication and high availability settings.

☁️ Virtual Private Server (VPS) with required networking resources in Google Cloud Platform (GCP). I use this VPS for hosting Uptime Kuma to monitor my homelab services.

📦 Bucket on Google Cloud Storage (GCS) for storing offsite backups.

🔒 Tailscale access control lists (ACLs). Thanks to data providers like tailscale_devices or tailscale_users I’m able to generate ACLs on the fly.

Ansible

Ansible roles define almost all the software I use in my homelab. I couldn’t imagine to maintain all that stuff manually. Here are some examples:

🛡️ Adguard DNS

📦 Apt Cacher NG

🛠️ Backup Ninja

🐳 Docker

📹 Frigate

📊 Grafana

📈 Grafana Agent

👴 Gramps

🌈 Hyperion NG

📸 Immich

🎥 Kodi

📂 Loki

📧 Mailrise

🐝 Mosqquitto

🔋 NUT

🌐 Omada Software Controller

📄 Paperless NGX

💾 Proxmox Backup Server

📈 Prometheus

🎵 Raspotify

🔄 RClone

🖥️ Samba

🔍 SearXNG

🎶 Shairport

📄 Stirling PDF

🔒 Tailscale

🚀 Traefik

📡 Transmission

📊 Uptime Kuma

🔐 Vaultwarden

🔍 Whoogle

📡 Zigbee2MQT

If you’re interested in how these services are set up in my homelab, you can explore the playbooks. Here are some examples: Proxmox hosts, Raspberry Pi, VPS.

Please note that I use a dynamic Ansible inventory for all my Linux containers. You can find more details in the inventory.proxmox.yml file. The static inventory includes only non-virtualized hosts such as Proxmox VE, Raspberry Pi, and VPS.

Conclusion

I hope you find my homelab setup useful and inspiring. If you have any questions, feel free to ask me on GitHub Discussions.

Yet another Dell Optiplex Micro

To build a Proxmox cluster, I needed an additional server. The existing Dell Optiplex Micro 3050 had served me well, so I decided to purchase a more powerful model this time. I opted for the Dell Optiplex Micro 5070, which comes with the following specifications:

• CPU Intel i5-9500T 2.2-3.7GHz, 6 cores
• GPU Intel® HD Graphics 630
• 32GB RAM
• 256 GB SSD, Samsung PM981 (NVMe, TLC)
• Built-in Gigabit Ethernet card
• USB 3.1 Gen 2 × 1 Type-C
• USB 3.1 Gen 1 × 5

Again for hosting VMs and containers, I mounted an enterprise-grade SSD: Intel DC S3610 1.6TB. This SSD is known for its high endurance and reliability, making it an excellent choice for a homelab environment where data integrity and performance are crucial. Despite having 44,123 power-on hours (~5 years), this model boasts an impressive Total Bytes Written (TBW) rating of 10.7PB. The current wear level is at 0%, indicating that the drive has plenty of life left and should continue to perform reliably for a long time. The Intel DC S3610’s high endurance is due to its use of Multi-Level Cell (MLC) NAND technology, which provides a good balance between performance, endurance, and cost.

Make a cluster quorum

I also bought a Dell Wyse 3040 thin client to use as a Proxmox QDevice to achieve cluster quorum. The Dell Wyse 3040 is equipped with a quad-core Intel Atom x5-Z8350 CPU, 2GB of RAM, and 8GB of eMMC storage.

Initially, I considered using a Raspberry Pi for this purpose. However, the Dell Wyse 3040 offered several advantages that made it a better option. Firstly, the built-in eMMC storage of the Wyse 3040 is more reliable and durable compared to the SD cards typically used in Raspberry Pi devices. SD cards are prone to wear and data corruption over time, especially under continuous read/write operations, which can be a significant drawback in a homelab environment where reliability is crucial. Additionally, the Dell Wyse 3040 is more affordable than a Raspberry Pi when considering the total cost, including necessary accessories such as a case, power supply, and storage.

Overall, the Dell Wyse 3040 provides a robust and reliable solution for maintaining cluster quorum in my Proxmox setup, ensuring high availability and seamless failover capabilities.

Below you can see the Dell Optiplex Micro 5070 and Dell Wyse 3040 disassembled for thermal paste replacement.

Bigger pipe

A faster network is crucial for ensuring efficient data transfer and reducing latency. It allows for quicker backups, faster VM migrations, and smoother operation of network-intensive applications.

So, I decided to pimp my network by hooking up both servers with some slick 2.5GbE network cards, while keeping my old 1GbE setup intact. I slapped in some RTL8125B network cards using PCIe M.2 A+E adapters, and boom, we’re in business.

This dedicated network is now the express lane for Proxmox cluster chatter and storage traffic.

1
2
3
4
5
6
7
8
$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  128 KByte (default)
------------------------------------------------------------
[  1] local 10.0.10.30 port 5001 connected with 10.0.10.31 port 41416 (icwnd/mss/irtt=14/1448/99)
[ ID] Interval       Transfer     Bandwidth
[  1] 0.0000-10.0134 sec  2.74 GBytes  2.35 Gbits/sec

Check out the final datacenter installed in my rack:

High availability

I chose not to use shared storage like a Synology NAS because it can become a single point of failure. Instead, replication offers a more resilient solution for my needs. ZFS really rocks! It provides robust data integrity, efficient snapshots, and seamless replication.

With all hardware in place, I started to configure the Proxmox cluster. I created a ZFS pool on both servers and defined a replication schedule to ensure that data is consistently mirrored between the servers. This setup allows for high availability and data redundancy, ensuring that my VMs can be quickly restored or migrated in case of hardware failure.

With replicated volumes, I can easily migrate VMs between servers and have a backup in case of hardware failure. Replication uses the 2.5GbE network for better performance. Thanks to the QDevice installed on the Dell Wyse 3040, Proxmox can automatically start VMs on the second server in case of the first server failure. I tested the failover when I pulled old Optiplex Micro 3050 for BIOS update. The VMs were automatically migrated to the second server and started without any issues!

External storage

In my homelab, Frigate stores video recordings on an external USB drive. This drive is connected to one of the servers and shared via NFS to the second server. From the Frigate container’s perspective, the drive is mounted as a local directory, which allows for easy migration of Frigate between servers.

Mounts in /etc/fstab on the server with the external drives:

1
2
UUID=... /mnt/usb1 ext4 defaults,nofail,x-systemd.device-timeout=10s 0 0
UUID=... /mnt/usb2 ext4 defaults,nofail,x-systemd.device-timeout=10s 0 0

Mounts in /etc/fstab on the server with the NFS share:

1
2
10.0.10.31:/mnt/usb1 /mnt/usb1 nfs4 defaults,rw,hard,rsize=1048576,wsize=1048576,timeo=300,retrans=2 0 0
10.0.10.31:/mnt/usb2 /mnt/usb2 nfs4 defaults,rw,hard,rsize=1048576,wsize=1048576,timeo=300,retrans=2 0 0

If the server with the local drive fails, I need to manually connect the drive to the second server and change the mount point from NFS to local. This manual intervention ensures that Frigate can continue to access the video recordings without interruption.

This setup provides a balance between flexibility and reliability, allowing me to handle server failures without requiring automated high availability for the external storage.

Backup

I migrated from VZDump to Proxmox Backup Server for several reasons. Firstly, Proxmox Backup Server offers deduplication, which significantly reduces the storage space required for backups. This is particularly beneficial in a homelab environment where storage efficiency is crucial. Additionally, Proxmox Backup Server provides faster backup and restore operations compared to VZDump, thanks to its optimized data handling and compression techniques.

Proxmox Backup Server runs as an LXC container and utilizes a dedicated ZFS pool for storing backups. This pool is replicated to the second server using Proxmox’s replication feature, ensuring that backup data is always available and protected against hardware failures.

To avoid a chicken-and-egg problem, VZDump is configured to back up the Proxmox Backup Server itself. This ensures that even if the Backup Server encounters issues, I can still restore it from VZDump backups.

Power management

Besides serving as the QDevice for Proxmox quorum, the Dell Wyse 3040 plays another crucial role in my homelab. It monitors the UPS connected via USB and runs the Linux NUT (Network UPS Tools) server. This setup ensures that both Optiplex Micro servers, configured as NUT clients, can be gracefully shut down in case of a power failure, preventing data loss and hardware damage.

Zigbee coordinator

Initially, I used a Sonoff ZBDongle-E USB stick connected to my server. However, this setup wasn’t ideal for a clustered environment since only one server could access the USB device at a time. To overcome this limitation, I upgraded to an SLZB-06 Zigbee CC2652P coordinator. This device connects via Ethernet, allowing the Zigbee2Mqtt container to access it regardless of which server it’s running on. I strategically placed the coordinator in the center of my home to ensure optimal coverage for all Zigbee devices. The device is powered by PoE, simplifying installation, and features a built-in web interface for easy configuration and monitoring.

Summary

Upgrading my homelab has been an incredibly rewarding experience. Starting with a single server allowed me to grasp the fundamentals before scaling up to a highly available cluster. My current setup boasts 10 high-performance cores, 64GB of RAM, and 800GB of replicated storage on enterprise-grade SSDs. This configuration ensures that even if one server fails, the other can seamlessly take over with minimal or no downtime.

The total cost of this homelab upgrade was approximately 500 USD, which included the Dell Optiplex Micro 5070, Dell Wyse 3040, Intel DC S3610 1.6TB SSD, two RTL8125B network cards, and a new Zigbee coordinator. Despite the enhancements, the power consumption of my homelab only increased by 30W, from 100W to 130W, keeping it efficient and manageable.

Modes

I introduced 3 modes to control how the house behaves. The modes improve readability and maintainability of my automations. They are implemented as simple boolean flags.

Away Mode

• When I arm all alarm partitions, automation enables “Away Mode”. This mode controls how the house behaves when I’m not at home.
• When I disarm all alarm partitions, automation disables “Away Mode”. It means I’m back home.

Night Mode

• Time based automation enables “Night Mode” when it’s time to sleep and disables it when it’s time to wake up. This mode controls how the house behaves when I’m sleeping.

Eco Mode

• Manual switch that controls heating and cooling systems. When I’m going to be away for a long time, I enable “Eco Mode” to save energy.

Automations

I organized automations in Home Assistant using packages. It helps me to keep configuration clean and organized.

When I’m writing this blog post, I have over 50 automations configured in Home Assistant and the number is growing. I will not list all of them here, but I will give you some examples.

CCTV

• Automation enables CCTV detections when I leave home, and disables it when I come back. All cameras belong to the same group, so I can enable/disable them all at once.
• Even if I’m at home, automation enables CCTV detections when “Night Mode” is active.
• If CCTV recognizes a person, automation sends a notification to my phones with the screenshot of detected entity.
• From the notification I’m able to open CCTV live feed using actionable notifications.
• I can also snooze detections for 5 minutes to avoid getting notifications when I’ve already knew who is at the yard. This automation uses timer for reliable snoozing.

Heating

• My heat pump uses heating curve to adjust heating power based on outside temperature. This is out of scope for my Home Assistant automations, but I have some automations to control house main thermostat. It enables me to adjust heating curve a bit based on my needs.
• Automation decreases heating temperature when “Eco Mode” is active.
• Automation turns off domestic hot water heating when “Eco Mode” is enabled.
• When I’m at home, automation increases heating one hour before “Night Mode” ends. It’s nice to wake up in warm house.
• Higher temperature back to normal 3 hours later.
• When I’m at home, automation increases heating 2 hours before “Night Mode” starts. It’s nice to take a shower in warm bathroom.
• Higher temperature back to normal 1 hour after “Night Mode” starts, it helps with drying floor and towels after shower.

Facade Lights

• Automation turns on facade lights in afternoon/evening when the Sun is 4 degrees below the horizon. “Night Mode” turns them off.
• To help me wake me up in the morning, automation turns on facade lights when “Night Mode” ends. When the Sun is 1 degree below the horizon, lights are turned off.
• If I’m not at home, facade lights automation is off.

Water

• Automation closes main water valve when I leave home, and opens it when I come back.
• The water valve remains open when dishwasher is running, and closes when it’s done.

Dishwasher

• When dishwasher is done, automation sends a notification to my son’s phone, unloading the dishwasher is his job.
• “Away Mode” disables dishwasher notifications.

Waste Collection

• Based on calendar, automation sends a notification day before waste collection to my phones. I can take out the trash in the evening.
• “Away Mode” disables waste collection notifications.

Power

• When I’m out of home, automation disables electric sockets on the terrace.
• When UPS is running on battery, automation sends a notification about power outage to my phones.
• When UPS battery is below 20%, NUT shuts down my servers gracefully.
• From time to time, I switch off Zigbee controlled socket in my rack to simulate power outage and test UPS. I’m going to automate this test in the future.

Windows

• When I’m leaving home, automation sends a notification if any window is open.
• When I’m out of home, automation sends a notification if any window opens.

Garage Gate

• When alarm garage partition is armed, automation disables garage gate switch and enables it again when alarm is disarmed.
• Because my Zigbee garage switch is not a momentary switch, automation turns it off after 5 seconds.

Fire

• When smoke detectors are triggered, besides the alarm sound, automation sends a notification to my phones.

Summary

I have a lot of fun with Home Assistant but I’m always ask my wife and son if they are happy with the automations 😀

I don’t use Flink on a daily basis, but I hoped to gain some inspiration that I could apply to my real-time data pipelines running on GCP Dataflow.

Keynotes

• Flink 2.0 announced during the first day of the conference, perfect timing
• Stephan Ewen with Feng Wang presented 15 years of the project history Apache Flink, which emerged around 2014, originally started as the Stratosphere project at German universities in 2009

• Kafka fails short in real-time streaming analytics

• Truly unified batch and streaming with Apache Paimon: Streaming Lakehouse.
• Fluss: Streaming storage for next-gen data analytics, it’s going to be open-sourced soon. Apologies for the low quality picture.

Revealing the secrets of Apache Flink 2.0

• Disaggregated state storage: Goodbye RocksDB
• External shuffle service: Welcome Apache Celeborn

• Streaming Lakehouse: An enabler for unified batch and streaming in an innovative way

• Breaking changes: While it’s unfortunate that the Scala API is deprecated, there’s a new extension available: Flink Scala API.
• PMC members mentioned that they’re going to modernize the Java Streaming API soon. It’s the oldest and hardest-to-maintain part of the Flink API.

Flink autoscaling: A year in review - performance, challenges and innovations

• Autoscaling example, simplified but self-explanatory.

• Challenges: Unfortunately, the speaker struggled with time management and couldn’t delve into the details. The key lesson for me: don’t scale up if there is no effect of scaling. Dataflow engineering team - can you hear me?

• Memory management in Flink, for me looks like a configuration and tuning nightmare. Flink autoscaling should help, see: FLIP-271.

Scaling Flink in the real world: Insights from running Flink for five years at Stripe

• The best session of the first day, in my opinion!
• I’m sure that Ben Augarten from Stripe knows how to manage Flink clusters and jobs at scale.

• With tight SLOs, there isn’t time for manual operations. If a job fails, roll back using the previously saved job graph. How do you decide if a job fails in a generic way? You should listen to the session.

• Use a proxy in front of the Kafka cluster to prevent jobs from getting stuck if a Kafka partition leader becomes unavailable. See: How Stripe keeps Kafka highly available across the globe

• Shared Zookeepers and shared Flink clusters can lead to issues with noisy neighbors and the propagation of failures. Extra operational costs are worth it to support system stability and performance.

Visually diagnosing operator state problems

• How to track the flow of data and identify where things go wrong?
• Can you inspect each late data record and figure out why it was late?
• Do you want to know what your state is before and after each step in your job?
• See also The Murky Waters of Debugging in Apache Flink: Is it a Black Box?
• Excellent logo, isn’t it?

Zero interference and resource congestion in Flink clusters with Kafka data sources

• Another session focused on current Kafka limitations
• Mitigation strategies for noisy neighbors in Kafka: quotas and cluster mirroring
• Introducing WarpStream: Confluent has acquired WarpStream
• Stateless, leaderless brokers
• Using object storage for state management: expect higher latency, but it should be acceptable for most use cases

From Apache Flink to Restate - Event processing for analytics and Transactions

• A new business idea from one of the Flink founders: shift the focus from analytical to transactional processing.
• Apply resilience and consistency lessons learned from building Flink to distributed transactional, RPC-based applications.

• In simple terms, it resembles an orchestrated Saga pattern
• Durable and reliable async/await
• See Why we built Restate

Enabling Flink’s Cloud-Native Future: Introducing ForSt DB in Flink 2.0

• The problem, local state doesn’t fit cloud native architecture.

• ForSt (for streaming) DB architecture

• Performance dropped 100x when RocksDB was replaced with object store as is.
• The new async API requires changes in all Flink operators!

• Asynchronous improves performance but introduces new challenge: ordering

• Slower than local RocksDB, but performance looks promising

Building Copilots with Flink SQL, LLMs and vector databases

• The most entertaining session of the conference, in my opinion
• How to adopt real-time analysis for non-technical users?

• Steffen Hoellinger invited us to conduct a POC together with Airy

• Hmm, some technical knowledge is still required 😀

• Key lesson: context is much more important than model
• Keep small workspaces to avoid hallucinations

• Flink SQL ML models, see: FLIP-437

Materialized Table - Making Your Data Pipeline Easier

• The most eye opening session
• Batch, incremental and real-time unification
• Backfill

• Freshness vs Cost
• Apply SET FRESHNESS = INTERVAL 1 HOUR and framework will do the rest
• Support for most SQL queries (without ORDER BY cause)

• Cool demo, materialized view freshness changed, Flink jobs rescheduled and BI dashboard updated in-place. Yet another scenario for backfill.
• Community version coming soon, see FLIP-435.

Event tracing

• Session based on IoT vehicle data in Mercedes-Benz.
• Apply OpenTelemetry for real-time data pipelines
• Tracing events sampling to avoid negative performance impact
• Kafka sources: extract tracing from headers
• Flink steps: attach spans to all events
• Kafka sinks: add tracing to headers
• In-summary: a lot of extra work

Summary

Attending the conference was a valuable experience, offering deep insights into the latest developments in Apache Flink. Here are my key takeaways:

• Listening to sessions about the challenges of Flink deployment and operations from the trenches made me appreciate the simplicity of Dataflow even more.
• I now believe in the potential of truly unified batch and streaming processing. FLIP-435 and the streaming lakehouse give hope that SET FRESHNESS could switch processing modes from batch, through incremental, to real-time.
• For high adoption of real-time analytics, consider using GenAI to hide the underlying data pipelines complexity.
• My general impression is that Kafka’s limitations in the cloud-native era have been confirmed.

You can say that this is a book about the basics, but these basics concern the construction of distributed systems, which isn’t an easy subject. For less experienced engineers, this book reduces the area of unknown unknowns, turning them into known unknowns. A lot of references allow them to deepen their knowledge as needed. Experienced engineers can treat this book as a refresher and identify areas where they need to supplement their knowledge.

My general observations during the lecture:

Clear and Structured Approach: The book explains complex principles in a clear and organized manner.

Wide Knowledge Coverage: It provides a broad overview rather than deep dives into specific topics.

Excellent References: The book includes many references to whitepapers, books, articles, and websites for further reading.

Below is my brief summary of the book:

• Rapid Business Evolution: The pace of business change is accelerating, necessitating that software systems keep up. Proper architecture is crucial for enabling software evolution.
• Embracing Incremental Changes: Software engineers can’t predict unknown unknowns. By embracing incremental changes to test hypotheses, similar to evolutionary processes, failed experiments are discarded while successful ones are retained.
• Short Release Cycles: Release cycles must be short. There is no time for manual, repetitive tasks; therefore, everything must be automated.

At the end of the book, there is a self-assessment section, which I partially summarized by writing my answers in this blog post.

What are the four dimensions that define software architecture

1. Architecture Characteristics: These define the success criteria of a system, such as performance, scalability, and security. They’re orthogonal to the system’s functionality.
2. Structure: This refers to the type of architecture style or styles used in the system, such as microservices, layered, or microkernel architectures.
3. Architecture Decisions: These are the rules and guidelines that dictate how a system should be constructed. They include choices about technologies, frameworks, and design patterns.
4. Design Principles: These are guidelines that help development teams make decisions about how to implement the system. They’re not hard-and-fast rules but rather best practices to follow.

What’s the difference between an architecture decision and a design principle

Architecture decisions are specific choices that define the architecture, while design principles are broader guidelines that influence those choices.

List the eight core expectations of a software architect

1. Make architecture decisions
2. Continually analyze the architecture
3. Keep current with latest trends
4. Ensure compliance with decisions
5. Diverse exposure and experience
6. Have business domain knowledge
7. Possess interpersonal skills
8. Understand and navigate politics

What’s the first law of software architecture

Everything is a tradeoff

List the three levels of knowledge in the knowledge triangle

1. Stuff you know
2. Stuff you know you don’t know
3. Stuff you don’t know you don’t know

What are the ways of remaining hands-on as an architect

• Coding regularly
• Side projects
• Pair programming
• Technical reading
• Training and courses
• Code reviews
• Prototyping and experimentation
• Mentoring

What’s meant by the term connascence

Describe the degree to which different parts of a system are interdependent

What’s the difference between static and dynamic connascence

• Static connascence occurs at the source code level, and developers can identify it by examining the code itself.
• Dynamic connascence is related to the runtime behavior of the system and developers can only identify it during execution, such as the order of calls.

What’s the strongest form of connascence

Connascence of Identity: This occurs when many components must reference the same entity, meaning any change to the identity of that entity requires changes across all components that reference it. For example, if the format or structure of the entity identifier is changed from a numeric to an alphanumeric code, you need to update every module that references this entity.

What’s the weakest form of connascence

Connascence of Name, this occurs when many components must agree on the name of an entity. For example, if the name of a method changes, all references to that method must also be updated. Usually straightforward to manage and refactor with IDE.

Which is preferred static or dynamic connascence

Static connascence refers to dependencies that the compiler can check at compile time, such as type checking and method signatures. Dynamic connascence, on the other hand, involves dependencies that are only checked at runtime, such as dynamic method calls or reflection.

In a code base, it’s preferable to use static connascence over dynamic connascence.

Give an example of an operational characteristic

• Availability
• Continuity (disaster recovery capability)
• Performance
• Recoverability
• Reliability/safety
• Robustness
• Scalability

Give an example of a structural characteristic

• Configurability
• Extensibility
• Installability
• Leverageability (ability to reuse common components)
• Localization
• Maintainability
• Portability
• Upgradeability

Give an example of a cross-cutting characteristic

• Accessibility
• Archivability (will the data need to be archived or deleted after a period of time)
• Authentication
• Authorization
• Legal
• Privacy
• Security
• Supportability
• Usability/achievability (level of training required for users to achieve their goals with the application)

Why it’s a good practice to limit the number of characteristics an architecture should support

• Avoiding complexity: Keep the architecture simpler and more understandable.
• Resource allocation: Ensure that resources are effectively allocated to the most critical aspects of the system.
• Trade-off management: Ensure that the system meets its most important goals without being overburdened by conflicting requirements.

What’s an architectural quantum

It refers to the smallest unit of an architecture that can be independently deployed and tested, encompassing all the necessary components to fulfill a specific business function.

What’s the difference between technical partitioning and domain partitioning

Technical partitioning focuses on technical roles, while domain partitioning focuses on business functionality. Domain partitioning often makes it easier to manage changes in business requirements, whereas technical partitioning can complicate changes due to inter-layer dependencies.

Under what circumstances would technical partitioning be a better choice over domain partitioning

• Small or simple applications
• Homogeneous teams, for example: front-end developers, back-end developers
• Legacy systems with technical partitioning
• Standardized processes, for example: compliance with certain regulations or industry standards
• Performance optimization

List the eight fallacies of distributed computing

1. The network is reliable
2. Latency is zero
3. Bandwidth is infinite
4. The network is secure
5. Topology doesn’t change
6. There is one administrator
7. Transport cost is zero
8. The network is homogeneous

What’s stamp coupling

Stamp coupling, also known as data-structured coupling, occurs when modules share a composite data structure and use only parts of it. This can lead to issues where changes in the unused parts of the data structure might affect the module that doesn’t need those parts.

What’s the difference between an open layer and a closed layer

An open layer permits requests to bypass it and directly access any layer below it. A closed layer requires that all requests pass through it before reaching any lower layer.

What’s the architecture sinkhole anti-pattern

The architecture sinkhole anti-pattern occurs when requests pass through multiple layers of an architecture without any significant processing or logic being applied at each layer. Essentially, the layers act as mere pass-throughs, adding unnecessary complexity and overhead without providing any real value.

Name the four types of filters and their purpose in pipeline architecture

• Input filters: Transform raw data into a form that’s suitable for later processing stages.
• Transform filters: Enrich the data or apply business logic.
• Output filter: Convert the data into the final format and write it to the destination.
• Error filters: Handle exceptions and errors that may occur during processing.

In what way does the pipeline architecture support modularity

• Separation of Concerns: Each step in the pipeline has a specific task, such as handling input, performing transformations, managing output, or handling errors.
• Reusability: Steps can be designed as reusable modules that can be plugged into different pipelines.
• Scalability: Individual steps can optimize, replace, or replicate without affecting other parts of the pipeline.
• Testing and debugging: Each step allows independent testing and debugging, making it easier to identify and fix issues.

What’s domain/architecture isomorphism

Principle where the structure of the software architecture closely mirrors the structure of the problem domain it’s designed to address. For example, in an operating system, the micro-kernel architecture’s minimal core will handle only the most fundamental aspects of the system (like process communication and basic I/O), while other services (like file systems and device drivers) will be handled by external modules.

What’s a primary difference between broker and mediator topologies

• In a broker topology, individual components or services communicate with each other through a message broker. Services are loosely coupled because they don’t need to know each other’s location or protocol.
• In a mediator topology, a mediator orchestrates and manages the communication between services. This approach is suitable for complex interactions that require coordination, transformation, and aggregation of many services.

What’s a primary aspect fo space-based architecture that differentiates in from other architecture styles

It splits both the processing and the storage across many servers. The Space Based Architecture pattern is designed to offer high scalability, fault-tolerance, and low-latency data access by storing data in memory across many nodes.

Name the four components that make up the virtualized middleware within a space-based architecture

• Message grid: Manages input request and session information
• Data grid: Manage the data replication between processing units when data updates occur
• Processing grid: Manages distributed request processing when there are many processing units, each handling a part of the application
• Deployment manager: Manages the dynamic startup and shutdown of processing units based on load conditions

What’s a difference between replicated cache and distributed cache

Replicated cache:

• Same data replicated to all nodes
• High fault tolerance, all nodes have the same data
• Low latency for read operations because each node has a full copy of the data
• Useful when read operations significantly outweigh write operations

Distributed cache:

• Stores different pieces of data on different cache nodes
• Lower compared to replicated cache
• Can be higher for read operations, since data might not be present on the local node
• Suitable for large datasets and when read and write operations are balanced