Skip to content

KPOps

Changelog

Changelog¶

9.0.0 - Release Date: [2024-12-20]¶

🏗️ Breaking changes¶

Introduce KPOps operation and manifest resources for deployment - #541
Drop support for Python 3.10 - #561
KPOps V9 - #558

🚀 Features¶

Introduce KPOps operation and manifest resources for deployment - #541
Define Pydantic model to representing Kubernetes manifest - #546
Manifest toSection with Strimzi KafkaTopic - #545
Manifest Kubernetes resources for destroy command - #552
Manifest Kubernetes resources for clean command - #559
Manifest Kubernetes resources for reset command - #563
KPOps V9 - #558

🐛 Fixes¶

Hide operation_mode from KPOps config - #571

📝 Documentation¶

Add documentation for operation-mode in KPOps - #565
Add migration guide v8-v9 - #562

🌀 Miscellaneous¶

Add topic manifestation of ProducerApps for reset command - #566

8.4.0 - Release Date: [2024-12-18]¶

🏭 Refactor¶

Create generic SerializeAsOptional type for Pydantic - #564

8.3.2 - Release Date: [2024-12-17]¶

🐛 Fixes¶

Fix allow optional resources requests and limits - #570

8.3.1 - Release Date: [2024-12-17]¶

🐛 Fixes¶

Fix Kubernetes memory not accepting decimal values - #568
Add ephemeral storage to Kubernetes resource requests and limits - #569

8.3.0 - Release Date: [2024-12-17]¶

🌀 Miscellaneous¶

Add Kubernetes affinity and tolerations to streams-bootstrap v2 values - #567

8.2.0 - Release Date: [2024-12-12]¶

🐛 Fixes¶

Update KPOps example snapshots and fix broken link to defaults.yaml - #560

📝 Documentation¶

Update KPOps example snapshots and fix broken link to defaults.yaml - #560

🌀 Miscellaneous¶

Bump streams-bootstrap to 3.1.0 - #557
Add Pydantic models for Kubernetes Affinity - #555

8.1.4 - Release Date: [2024-12-09]¶

🐛 Fixes¶

Fix kpops --version - #551

🌀 Miscellaneous¶

Trim Helm name override for Producer CronJob to 52 characters - #550

8.1.3 - Release Date: [2024-12-05]¶

🌀 Miscellaneous¶

Remove repeated defaults from streams-bootstrap values - #547

8.1.2 - Release Date: [2024-12-04]¶

🌀 Miscellaneous¶

Convert all values of Kafka connector and topic config to string - #544

8.1.1 - Release Date: [2024-12-02]¶

🐛 Fixes¶

Fix files field value type in Streamsboostrap component - #542
Fix: Use enum values when dumping models - #543

8.1.0 - Release Date: [2024-10-25]¶

🧪 Dependencies¶

Upgrade typer to support union types - #533

🌀 Miscellaneous¶

Extend StreamsBootstrap model - #534

8.0.1 - Release Date: [2024-08-22]¶

📝 Documentation¶

Fix changelog in docs - #532

8.0.0 - Release Date: [2024-08-21]¶

🏗️ Breaking changes¶

Make KafkaApp responsible for deploying/cleaning streams bootstrap components - https://github.com/bakdata/kpops/pull/522
Rename role to label - https://github.com/bakdata/kpops/pull/525
Fix Pyright warning about type override without default value - https://github.com/bakdata/kpops/pull/524
Remove -v3 suffix and suffix old streams bootstrap with -v2 - https://github.com/bakdata/kpops/pull/526

🚀 Features¶

Add support for streams-bootstrap v3 - https://github.com/bakdata/kpops/pull/519

🏭 Refactor¶

Rename role to label - https://github.com/bakdata/kpops/pull/525
Fix Pyright warning about type override without default value - https://github.com/bakdata/kpops/pull/524

7.1.0 - Release Date: [2024-08-15]¶

🧪 Dependencies¶

Update pytest - #527

🏭 Refactor¶

Improve incomplete type hints - #515
Fallback to user defined model when the validation of cluster model fails - #521
Replace kubernetes-asyncio with lightkube - #517

🌀 Miscellaneous¶

Fix incorrect parameter type annotation - #523

7.0.0 - Release Date: [2024-07-23]¶

🏗️ Breaking changes¶

Automatic loading of namespaced custom components - #500
Call destroy from inside of reset or clean - #501
Rename app field - #506

🚀 Features¶

clean/reset streams-bootstrap components with cluster values - #498

🏭 Refactor¶

Call destroy from inside of reset or clean - #501
Rename app field - #506

6.1.0 - Release Date: [2024-07-09]¶

🚀 Features¶

Add image tag field to streams-bootstrap app values - #499
Delete ignored keys from diff - #510

🏭 Refactor¶

Improve dataclass instance check - #507

6.0.2 - Release Date: [2024-07-04]¶

📝 Documentation¶

Generate developer docs for Python API - #503

🌀 Miscellaneous¶

Update codeowners - #504

6.0.1 - Release Date: [2024-06-12]¶

🐛 Fixes¶

Fix connector resetter offset topic - #497

6.0.0 - Release Date: [2024-06-06]¶

🏗️ Breaking changes¶

KPOps 6.0.0 - #496

🚀 Features¶

KPOps 6.0.0 - #496

🏭 Refactor¶

KPOps 6.0.0 - #496

5.1.1 - Release Date: [2024-05-22]¶

🐛 Fixes¶

Add YAML separator (---) to stdout - #491

5.1.0 - Release Date: [2024-05-22]¶

🌀 Miscellaneous¶

Add computed field for Helm release name and name override - #490

5.0.1 - Release Date: [2024-05-15]¶

🐛 Fixes¶

Fix missing await on Kubernetes API - #488

5.0.0 - Release Date: [2024-05-02]¶

🏗️ Breaking changes¶

Allow custom timeout for external services - #485

🌀 Miscellaneous¶

Update examples for v4 - #486

4.2.1 - Release Date: [2024-04-25]¶

🚀 Features¶

Add support for cleaning StatefulSets with PVCs - #482

4.2.0 - Release Date: [2024-04-25]¶

🏭 Refactor¶

Improve type annotations for parallel pipeline jobs - #476

🌀 Miscellaneous¶

Update Ruff - #475
Set Pyright to warn on unknown types - #480
Quiet faker debug logs in tests - #483
Add pyright matcher - #481

4.1.2 - Release Date: [2024-03-11]¶

📝 Documentation¶

fix(docs): Correct from.components.<component-name>.type to input - #473

4.1.1 - Release Date: [2024-03-11]¶

🐛 Fixes¶

Fix import errors - #472

🏭 Refactor¶

Fix import errors - #472

🌀 Miscellaneous¶

Update httpx - #471

4.1.0 - Release Date: [2024-03-07]¶

📝 Documentation¶

Document precedence between env vars and config.yaml - #465

🌀 Miscellaneous¶

Create init command - #394

4.0.2 - Release Date: [2024-03-04]¶

📝 Documentation¶

Reference editor plugin for Neovim in docs - #464

🌀 Miscellaneous¶

Add support for Python 3.12 - #467
Update Pyright - #468
Remove package classifiers that are automatically assigned by Poetry - #469
Validate autoscaling mandatory fields when enabled - #470

4.0.1 - Release Date: [2024-02-29]¶

🐛 Fixes¶

Set supported Python cutoff to 3.11 - #466

4.0.0 - Release Date: [2024-02-27]¶

🏗️ Breaking changes¶

Distribute defaults across multiple files - #438

🚀 Features¶

Distribute defaults across multiple files - #438

3.2.4 - Release Date: [2024-02-26]¶

🐛 Fixes¶

Fix docs CI to include the latest changes to a tagged version in the changelog - #459
Fix tempfile creation - #461
Fix symbolic link to CONTRIBUTING.md and parallel option in action.yaml - #462

🏭 Refactor¶

Refactor Kafka topics - #447
Refactor PipelineGenerator to use component ids - #460

📝 Documentation¶

Fix docs CI to include the latest changes to a tagged version in the changelog - #459

3.2.3 - Release Date: [2024-02-19]¶

🐛 Fixes¶

Trim and hash Helm name override to 63 characters - #456

3.2.2 - Release Date: [2024-02-12]¶

🐛 Fixes¶

Fix nested substitution - #451

3.2.1 - Release Date: [2024-02-08]¶

🐛 Fixes¶

Fix order of pipeline steps for clean/reset - #450
Fix substitution - #449
Fix cleaner inheritance, parent model should be aliased during instantiation - #452

🏭 Refactor¶

Simplify execution graph logic - #446

3.2.0 - Release Date: [2024-02-01]¶

🚀 Features¶

Refactor pipeline filter and add to public API - #405

🏭 Refactor¶

Refactor enrichment using Pydantic model validator - #444
Refactor pipeline filter and add to public API - #405

📝 Documentation¶

Improve Sphinx docs highlighting using RST markup - #443

3.1.0 - Release Date: [2024-01-30]¶

🚀 Features¶

Add support for pipeline steps parallelization - #312

🐛 Fixes¶

Update poetry publish workflow version to latest - #430

🏭 Refactor¶

Simplify loading of defaults - #435

🌀 Miscellaneous¶

Add custom PascalCase to snake_case alias generator - #436
Add parallel flag support to kpops runner - #439

3.0.2 - Release Date: [2024-01-23]¶

🐛 Fixes¶

Fix Helm diff output - #434

📝 Documentation¶

Add step for submodule initialization on the docs - #431

🌀 Miscellaneous¶

Add message if examples git submodule is not initialized - #432
Update type annotation for deserialized pipeline - #433

3.0.1 - Release Date: [2024-01-19]¶

🐛 Fixes¶

Fix broken doc link - #427
Add warning log if SR handler is disabled but URL is set - #428

📝 Documentation¶

Update docs of word-count example for v3 & new folder structure - #423
Move ATM fraud to examples repo - #425
Fix broken doc link - #427

🌀 Miscellaneous¶

Update pydantic dependency - #422
Add git submodule instructions to the contributing.md - #429

3.0.0 - Release Date: [2024-01-17]¶

🏗️ Breaking changes¶

Move GitHub action to repository root - #356
Make Kafka REST Proxy & Kafka Connect hosts default and improve Schema Registry config - #354
Create HelmApp component - #370
Change substitution variables separator to . - #388
Refactor pipeline generator & representation - #392
Define custom components module & pipeline base dir globally - #387
Use hash and trim long Helm release names instead of only trimming - #390
Refactor generate template for Python API usage - #380
Namespace substitution vars - #408
Refactor streams-bootstrap cleanup jobs as individual HelmApp - #398
Refactor Kafka Connector resetter as individual HelmApp - #400
Fix wrong Helm release name character limit - #418

🚀 Features¶

Allow overriding config files - #391
Generate defaults schema - #402

🐛 Fixes¶

Fix missing component type in pipeline schema - #401
Fix enrichment of nested Pydantic BaseModel - #415
Fix wrong Helm release name character limit - #418
Update release workflow template to support custom changelog file path - #421

🧪 Dependencies¶

Migrate to Pydantic v2 - #347

🏭 Refactor¶

Make Kafka REST Proxy & Kafka Connect hosts default and improve Schema Registry config - #354
Migrate to Pydantic v2 - #347
Refactor pipeline generator & representation - #392
Use hash and trim long Helm release names instead of only trimming - #390
Refactor Helm nameOverride - #397
Mark component type as computed Pydantic field - #399
Refactor generate template for Python API usage - #380
Support multiple inheritance for doc generation - #406
Refactor streams-bootstrap cleanup jobs as individual HelmApp - #398
Refactor Kafka Connector resetter as individual HelmApp - #400

📝 Documentation¶

Move GitHub action to repository root - #356
Create HelmApp component - #370
Update docs for substitution variable usage in v3 - #409
Support multiple inheritance for doc generation - #406
Update docs for v3 - #416
Update tests resources - #417
Summarize all breaking changes in diffs at the top of the migration guide - #419

🌀 Miscellaneous¶

Replace black with ruff - #365
Add toml formatter to dprint - #386
Add malva to dprint - #385
Update KPOps runner with the new options - #395
Fix KPOps action to get package from testPyPI - #396
KPOps 3.0 - #420

2.0.11 - Release Date: [2023-10-24]¶

🐛 Fixes¶

Fix early exit upon Helm exit code 1 - #376
Fix docs setup page list indentation - #377

📝 Documentation¶

Migrate deprecated mkdocs-material-extensions - #378
Fix docs setup page list indentation - #377
Exclude resources from docs search - #371

2.0.10 - Release Date: [2023-10-12]¶

🌀 Miscellaneous¶

Fix environment variables documentation generation - #362
Introduce ruff - #363
Print details on connector name mismatch error - #369
Enable transparent OS environment lookups from internal environment - #368

2.0.9 - Release Date: [2023-09-19]¶

🐛 Fixes¶

Fix Kafka connect config name for deletion - #361

📝 Documentation¶

Fix link to kpops-examples - #357

2.0.8 - Release Date: [2023-09-06]¶

🐛 Fixes¶

Fix config.yaml overriding environment variables - #353

🏭 Refactor¶

Refactor component prefix & name - #326
Remove unnecessary condition during inflate - #328

2.0.7 - Release Date: [2023-08-31]¶

🐛 Fixes¶

Print only rendered templates when --template flag is set - #350

📝 Documentation¶

Add migration guide - #352

2.0.6 - Release Date: [2023-08-30]¶

🏭 Refactor¶

Simplify deployment with local Helm charts - #349

2.0.5 - Release Date: [2023-08-30]¶

🐛 Fixes¶

Fix versioning of docs when releasing - #346

2.0.4 - Release Date: [2023-08-29]¶

🐛 Fixes¶

Fix GitHub ref variable for pushing docs to main branch - #343

📝 Documentation¶

Add dprint as the markdown formatter - #337
Publish pre-release docs for PRs & main branch - #339
Align docs colours - #345

🌀 Miscellaneous¶

Exclude abstract components from pipeline schema - #332

2.0.3 - Release Date: [2023-08-24]¶

🐛 Fixes¶

Fix GitHub action error in non-Python projects - #340

🌀 Miscellaneous¶

Lint GitHub action - #342

2.0.2 - Release Date: [2023-08-23]¶

📝 Documentation¶

Add version dropdown to the documentation - #336
Break the documentation down into smaller subsection - #329

2.0.1 - Release Date: [2023-08-22]¶

🐛 Fixes¶

Fix optional flags in GitHub action - #334

2.0.0 - Release Date: [2023-08-17]¶

🏗️ Breaking changes¶

Remove camel case conversion of internal models - #308
Derive component type automatically from class name - #309
Refactor input/output types - #232
v2 - #321

🚀 Features¶

Automatically support schema generation for custom components - #307
Derive component type automatically from class name - #309

🏭 Refactor¶

Refactor input/output types - #232

📝 Documentation¶

Fix editor integration example in docs - #273

1.7.2 - Release Date: [2023-08-16]¶

🏭 Refactor¶

Refactor Kafka Connect handler - #322

📝 Documentation¶

Add KPOps Runner GitHub Action to the documentation - #325
Remove :type and :rtype from docstrings - #324

1.7.1 - Release Date: [2023-08-15]¶

📝 Documentation¶

Modularize and autogenerate examples for the documentation - #267
Update the variable documentation - #266

1.7.0 - Release Date: [2023-08-15]¶

🚀 Features¶

Add flag to exclude pipeline steps - #300

1.6.0 - Release Date: [2023-08-10]¶

🏭 Refactor¶

Refactor handling of Helm flags - #319

1.5.0 - Release Date: [2023-08-10]¶

🚀 Features¶

Refactor Helm wrapper and add --set-file flag - #311

🏭 Refactor¶

Refactor Helm wrapper and add --set-file flag - #311
Set default for ToSection topics - #313
Annotate types for ToSection models mapping - #315

🌀 Miscellaneous¶

Check Poetry lock file consistency - #316

1.4.0 - Release Date: [2023-08-02]¶

🐛 Fixes¶

Validate unique step names - #292

🏭 Refactor¶

Order PipelineComponent fields - #290
Migrate requests to httpx - #302
Refactor CLI using dtyper - #306

🌀 Miscellaneous¶

Update Black - #294
Fix vulnerability in mkdocs-material - #295
Move breaking changes section upper in the change log config - #287

1.3.2 - Release Date: [2023-07-13]¶

🐛 Fixes¶

Exclude Helm tests from dry-run diff - #293

1.3.1 - Release Date: [2023-07-11]¶

🏭 Refactor¶

Remove workaround for pipeline steps - #276

🌀 Miscellaneous¶

Update codeowners - #281
Reactivate Windows CI - #255
Downgrade Poetry version on the Windows CI pipeline - #286
Set ANSI theme for output of kpops generate - #289

1.3.0 - Release Date: [2023-07-07]¶

🏭 Refactor¶

Plural broker field in pipeline config - #278

📝 Documentation¶

Update KPOps runner readme for dev versions - #279

🏗️ Breaking changes¶

Plural broker field in pipeline config - #278

🌀 Miscellaneous¶

Add breaking changes section to change log config - #280

1.2.4 - Release Date: [2023-06-27]¶

🌀 Miscellaneous¶

Update changelog action to contain miscellaneous PRs - #269

1.2.3 - Release Date: [2023-06-22]¶

🐛 Fixes¶

Refactor custom component validation & hide field from kpops output - #265

🏭 Refactor¶

Refactor custom component validation & hide field from kpops output - #265

🌀 Miscellaneous¶

1.2.2 - Release Date: [2023-06-21]¶

🌀 Miscellaneous¶

Create workflow to lint CI - #260
Fix update docs when releasing - #261
Rename change log message for uncategorized issues - #262

1.2.1 - Release Date: [2023-06-21]¶

Uncategorized

Fix update docs in release workflow - #258

1.2.0 - Release Date: [2023-06-21]¶

🚀 Features¶

Add helm repo update <repo-name> for Helm >3.7 - #239

🐛 Fixes¶

add --namespace option to Helm template command - #237
Add missing type annotation for Pydantic attributes - #238
Fix helm version check - #242
Fix Helm Version Check - #244
Fix import from external module - #256

🏭 Refactor¶

Remove enable option from helm diff - #235
Refactor variable substitution - #198

Uncategorized

Add background to docs home page - #236
Update Poetry version in CI - #247
Add pip cache in KPOps runner action - #249
Check types using Pyright - #251
Remove MyPy - #252
Disable broken Windows CI temporarily - #253
Update release and publish workflows - #254
Fix release & publish workflows - #257