feat(pricing): expand gateway max price to set per capability/model id #3116

ad-astra-video · 2024-08-03T14:05:18Z

This PR allows setting prices per capability and model id.

Current Implementation

Gateways currently use one MaxPrice set at launch with maxPricePerUnit flag and can be adjusted with the setBroadcastConfig endpoint of the cli webserver while the gateway is running.

Enhancement Requested

Gateways on the AI subnet need the ability to filter orchestrators on price for different pipelines/model ids. Transcoding would also benefit from this when different encoding codecs are enabled on the network. The single MaxPrice needs to be updated to enable setting prices per capability (e.g. text-to-image and image-to-video) separately as well as allow the ability to price models within the capabilities separately. The single MaxPrice will function as the overall default if no prices are set for specific pipelines and/or models. The separate pricing needs to also allow setting a price per capability for all models in a capability.

Proposed implementation Changes

A maxPricePerCapability launch flag is added to accept json configuration of maximum prices for a capability/model. The json configuration uses the maxPricePerCapability flag configuration to provide consistency in how prices are set on the network. The server.BroadcastCfg is updated on the Gateway to store prices per capability/model id that can then be queried to provide the appropriate MaxPrice to use when selecting orchestrators in the selection algorithm. The server.BroadcastCfg is also extended to add new functions for GetCapabilitiesMaxPrice and GetCapabilityMaxPrice to provide ability to get correct price with provided capabilities of the request. The GetCapabilitiesMaxPrice returns a total MaxPrice for the capabilities included in the request. If no specific max prices is set for the capabilities requested, the MaxPrice is used. GetCapabilityMaxPrice returns the max price set for that capability/model id or zero if not set.

The base MaxPrice and SetMaxPrice functions are updated to set pricing of H264 encoding capability to allow extending this to provide different prices for different codecs in the future. ModelID is not used for transcoding so expected to always be "default".

eliteprox

Great work @ad-astra-video! I tested -maxPricePerCapability with AI model pricing and it works correctly.

I also regression tested -maxPricePerUnit to ensure price controls still work with transcoding. Using the H264 capability as the default price index is a creative approach that should allow us to set prices per codec one day in the future.

I tested livepeer_cli with both options 16, 15 and found the price changes also work correctly there. I made a few edits to setMaxPriceForCapability to prevent a runtime error and updated the test TestMaxPricePerCapability, see my suggestions below.

Overall, changes LGTM!

cmd/livepeer/starter/starter_test.go

server/handlers.go

Co-authored-by: John | Elite Encoder <john@eliteencoder.net>

ad-astra-video · 2024-08-16T04:34:14Z

@eliteprox @rickstaa I agree with all suggestions and committed the suggested changes.

eliteprox

Changes LGTM. Approved pending review from @rickstaa

victorges · 2024-08-16T21:11:21Z

cmd/livepeer/starter/starter.go

+		if p.Gateway == "" {
+			p.Gateway = "default"
+		}


What is this field used for if, from what I understood, this is a config on the gateway itself? I thought this would make sense on an orchestrator config to set prices for specific gateways instead.

Removed Gateway field for now. There was a time that this logic was going to be re-used for the Orchestrator as well to set price per gateway/pipeline/model_id but I think we found a way to re-use the aiModels.json config parsing to set price per gateway at the Orchestrator. Can always add it back in if needed.

Confirmed test still passed.

victorges · 2024-08-16T21:11:58Z

cmd/livepeer/starter/starter.go

+	prices := make([]ModelPrice, len(pricesSet.CapabilitiesPrices))
+	for i, p := range pricesSet.CapabilitiesPrices {
+		if p.Gateway == "" {
+			p.Gateway = "default"


nit: Could be nice to support this defaulting for the model_id field as well. Then one can specify only the pipeline for it to be the default price for the pipeline

This is a good idea. I removed the p.Gateway check and changed it to setting p.ModelID to 'default' if not set specifically.

victorges · 2024-08-16T21:14:55Z

core/ai.go

+	//no capability description matches name
+	return Capability_Unused


I wonder if this should be an explicit error instead (or the caller check this Unused value) to avoid silent bad configurations. Otherwise one might be missing a configuration that they swear they are making on the ir JSON config file which could have bad implications (e.g. paying more than they were willing to). In general I think it's better to fail explicitly for bad configs than to ignore them.

I agree that it would be better to update this to return (Capability, error) to make it more obvious it should be checked by the caller.

Updated and added a test for this function in ai_test.go

…, assign modelID to 'default' if not set, add/update tests

…base

ad-astra-video · 2024-08-17T13:44:04Z

@victorges @rickstaa updates made for review comments. Small merge conflict on a log line that looked the exact same resolved as well.

rickstaa · 2024-08-19T13:21:40Z

@ad-astra-video the code looks good 🚀. However I was unable to build this on my system 🤔. Could you check if you pushed the latest changes?

GO111MODULE=on CGO_ENABLED=1 CC="" CGO_CFLAGS="" CGO_LDFLAGS=" " go build -o "./" -tags "mainnet" -ldflags="-X github.com/livepeer/go-livepeer/core.LivepeerVersion=0.7.6-cdd74620" cmd/livepeer/*.go
# github.com/livepeer/go-livepeer/server
server/ai_session.go:206:3: undefined: Models
server/ai_session.go:208:29: capabilityConstraints[cap].Models undefined (type *"github.com/livepeer/go-livepeer/core".PerCapabilityConstraints has no field or method Models)
server/ai_session.go:209:14: undefined: core.NewCapabilitiesWithConstraints
server/broadcast.go:102:50: netCaps.CapabilityConstraints undefined (type *"github.com/livepeer/go-livepeer/net".Capabilities has no field or method CapabilityConstraints)
make: *** [Makefile:100: livepeer] Error 1

ad-astra-video · 2024-08-19T19:14:27Z

@rickstaa updated and re-tested this.

docker image build available at docker pull adastravideo/go-livepeer:ai-video-set-max-price-per-capability-rebase

rickstaa · 2024-08-22T15:01:59Z

cmd/livepeer/livepeer.go

@@ -135,6 +135,7 @@ func parseLivepeerConfig() starter.LivepeerConfig {
 	cfg.OrchPerfStatsURL = flag.String("orchPerfStatsUrl", *cfg.OrchPerfStatsURL, "URL of Orchestrator Performance Stream Tester")
 	cfg.Region = flag.String("region", *cfg.Region, "Region in which a broadcaster is deployed; used to select the region while using the orchestrator's performance stats")
 	cfg.MaxPricePerUnit = flag.String("maxPricePerUnit", *cfg.MaxPricePerUnit, "The maximum transcoding price per 'pixelsPerUnit' a broadcaster is willing to accept. If not set explicitly, broadcaster is willing to accept ANY price. Can be specified in wei or a custom currency in the format <price><currency> (e.g. 0.50USD). When using a custom currency, a corresponding price feed must be configured with -priceFeedAddr")
+	cfg.MaxPricePerCapability = flag.String("maxPricePerCapability", *cfg.MaxPricePerCapability, `json list of prices per capability/model or path to json config file. Use "model_id": "default" to price all models in a pipeline the same. Example: {"capabilities_prices": [{"pipeline": "text-to-image", "model_id": "stabilityai/sd-turbo", "priceperunit": 1000, "pixelsperunit": 1}, {"pipeline": "upscale", "model_id": "default", priceperunit": 1200, "pixelsperunit": 1}]}`)


Like the format! Do we want to add the currency field in there as well as an example?

Can we change this to snake case to keep the JSON convention. For this we can also change the PricePerGateway in this format (see https://linear.app/livepeer-ai/issue/AI-574/change-priceperbroadcaster-json-input-keys-to-recommended-conventions).

Updated. Will update pricePerGateway to same in separate PR.

rickstaa · 2024-08-22T16:03:27Z

monitor/census.go

+		mMinGasPrice                   *stats.Float64Measure
+		mMaxGasPrice                   *stats.Float64Measure
+		mTranscodingPrice              *stats.Float64Measure
+		mTranscodingPricePerCapability *stats.Float64Measure


Can we name it PricePerCapability?

rickstaa · 2024-08-22T16:16:30Z

server/broadcast.go

+	return price
+}
+
+func (cfg *BroadcastConfig) GetCapabilityMaxPrice(cap core.Capability, modelID string) *big.Rat {


This one can be lowercase since it is not used outside this file :).

This commit removes the BroadcastConfig initialisation method out of the starter to keep all logic inside the server module.

rickstaa · 2024-08-22T17:18:28Z

server/broadcast.go

+
+	return &BroadcastConfig{
+		maxPricePerCapability: map[core.Capability]map[string]*core.AutoConvertedPrice{
+			core.Capability_H264: models,


As discussed in our chat this can be core.Capability_Unused.

…base

rickstaa · 2024-08-23T15:49:51Z

cmd/livepeer/starter/starter.go

@@ -1810,6 +1837,58 @@ func getGatewayPrices(gatewayPrices string) []GatewayPrice {
 	return prices
 }

+type ModelPrice struct {
+	Gateway       string


I think this field is a left over of another feature 🤔? Couldn't see it used.

This commit improves the parsing of the maxPricePerCapability config so that it doesn't throw nil errors and defaults to the regular transcoding values. It also throws clear log statments when config values are incorrect.

This commit simplifies the PipelineToCapability because we don't use unicode in our capability naming.

This commit applies some minor code improvements.

This commit fixes small typo in the 'price_per_capability' metric.

This commit improves the defaults used in the setBroadcastMaxPricePerCapability CLI option.

This commit ensures that we unsubscribe to the priceFeed when a model maxPrice is changed.

This commit applies some common go code conventions to the NewAISessionSelector function.

This commit optimizes the code a bit.

….com:ad-astra-video/go-livepeer into ai-video-set-max-price-per-capability-rebase

rickstaa

LGTM thanks!

ad-astra-video added 2 commits August 3, 2024 07:43

enable setting max price per capability by gateway

f2e8d47

fix capability price log line on startup

6bf4da5

github-actions bot added the AI Issues and PR related to the AI-video branch. label Aug 3, 2024

ad-astra-video added 2 commits August 6, 2024 06:08

fix setting max price from cli and webserver endpoint

3fa5448

fix price convert to int64 when numerator or denominator overflows int64

2a94ffa

ad-astra-video marked this pull request as ready for review August 7, 2024 11:25

ad-astra-video requested a review from rickstaa as a code owner August 7, 2024 11:25

rickstaa deleted the branch livepeer:ai-video August 7, 2024 20:53

rickstaa closed this Aug 7, 2024

rickstaa reopened this Aug 7, 2024

rickstaa deleted the branch livepeer:ai-video August 10, 2024 06:53

rickstaa closed this Aug 10, 2024

rickstaa reopened this Aug 10, 2024

rickstaa changed the base branch from ai-video-rebase to ai-video August 10, 2024 15:25

eliteprox requested changes Aug 14, 2024

View reviewed changes

ad-astra-video and others added 2 commits August 15, 2024 23:32

Apply suggestions from code review

055416d

Co-authored-by: John | Elite Encoder <john@eliteencoder.net>

Apply suggestions from code review (missed one)

931d84a

Co-authored-by: John | Elite Encoder <john@eliteencoder.net>

eliteprox approved these changes Aug 16, 2024

View reviewed changes

victorges reviewed Aug 16, 2024

View reviewed changes

victorges approved these changes Aug 16, 2024

View reviewed changes

ad-astra-video and others added 2 commits August 17, 2024 07:44

updates for review comments: add error return to PipelineToCapability…

ac5e3b1

…, assign modelID to 'default' if not set, add/update tests

Merge branch 'ai-video' into ai-video-set-max-price-per-capability-re…

cdd7462

…base

ad-astra-video added a commit to ad-astra-video/go-livepeer that referenced this pull request Aug 18, 2024

add PipelineToCapability from PR livepeer#3116

56f807b

ad-astra-video added 4 commits August 19, 2024 09:15

update capabilities created for selectors

9e5189c

update GetCapabilitiesMaxPrice

c235c7f

update max price per capability handlers test to initialize bradcastcfg

e007090

fix server tests for api changes

c54b99f

ad-astra-video added a commit to ad-astra-video/go-livepeer that referenced this pull request Aug 21, 2024

add PipelineToCapability from PR livepeer#3116

b45c19b

rickstaa reviewed Aug 22, 2024

View reviewed changes

rickstaa added 2 commits August 22, 2024 18:36

refactor: remove BroadcastConfig init method

0916059

This commit removes the BroadcastConfig initialisation method out of the starter to keep all logic inside the server module.

fixup! refactor: remove BroadcastConfig init method

7f99222

rickstaa reviewed Aug 22, 2024

View reviewed changes

ad-astra-video and others added 6 commits August 22, 2024 12:24

update for review comments

fe75050

fix newBroadCastConfig

cb71410

Merge branch 'ai-video' into ai-video-set-max-price-per-capability-re…

d2c04bf

…base

fix price update log line to not overwrite the capName with last set

9a99d63

set model id for price change log

db0e15d

move capName and modelID below err check

d7d0065

rickstaa reviewed Aug 23, 2024

View reviewed changes

rickstaa and others added 11 commits August 23, 2024 19:22

feat(ai): improve maxPricePerCapability parsing

29334a8

This commit improves the parsing of the maxPricePerCapability config so that it doesn't throw nil errors and defaults to the regular transcoding values. It also throws clear log statments when config values are incorrect.

refactor(ai): refactor PipelineToCapability method

625bbd5

This commit simplifies the PipelineToCapability because we don't use unicode in our capability naming.

fixup! refactor(ai): refactor PipelineToCapability method

9ea6441

refactor(ai): apply some small code improvements

caa9037

This commit applies some minor code improvements.

refactor(ai): fix 'price_per_capability' typo

ebe032a

This commit fixes small typo in the 'price_per_capability' metric.

refactor(ai): improve setBroadcastMaxPricePerCapability defaults

2eaf8ae

This commit improves the defaults used in the setBroadcastMaxPricePerCapability CLI option.

fix(ai): handle priceFeed subscription memory leak

a4ae9e5

This commit ensures that we unsubscribe to the priceFeed when a model maxPrice is changed.

refactor(ai): apply code conventions

15fcdd4

This commit applies some common go code conventions to the NewAISessionSelector function.

fix GetCapabilitiesMaxPrice to check for nil capabilities

a655f45

refactor(ai): optimize code

174cf32

This commit optimizes the code a bit.

Merge branch 'ai-video-set-max-price-per-capability-rebase' of github…

649d6fd

….com:ad-astra-video/go-livepeer into ai-video-set-max-price-per-capability-rebase

rickstaa approved these changes Aug 23, 2024

View reviewed changes

rickstaa merged commit 4f3947d into livepeer:ai-video Aug 23, 2024
6 of 8 checks passed

rickstaa mentioned this pull request Sep 23, 2024

selection algorithm transcoding conflict patch #3181

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(pricing): expand gateway max price to set per capability/model id #3116

feat(pricing): expand gateway max price to set per capability/model id #3116

ad-astra-video commented Aug 3, 2024 •

edited

Loading

eliteprox left a comment

ad-astra-video commented Aug 16, 2024

eliteprox left a comment •

edited

Loading

victorges Aug 16, 2024

ad-astra-video Aug 17, 2024 •

edited

Loading

victorges Aug 16, 2024

ad-astra-video Aug 17, 2024

victorges Aug 16, 2024

ad-astra-video Aug 17, 2024

ad-astra-video commented Aug 17, 2024 •

edited

Loading

rickstaa commented Aug 19, 2024

ad-astra-video commented Aug 19, 2024

rickstaa Aug 22, 2024

rickstaa Aug 22, 2024

ad-astra-video Aug 22, 2024

rickstaa Aug 22, 2024

ad-astra-video Aug 22, 2024

rickstaa Aug 22, 2024

ad-astra-video Aug 22, 2024

rickstaa Aug 22, 2024

ad-astra-video Aug 22, 2024

rickstaa Aug 23, 2024

rickstaa left a comment

		//no capability description matches name
		return Capability_Unused

feat(pricing): expand gateway max price to set per capability/model id #3116

feat(pricing): expand gateway max price to set per capability/model id #3116

Conversation

ad-astra-video commented Aug 3, 2024 • edited Loading

eliteprox left a comment

Choose a reason for hiding this comment

ad-astra-video commented Aug 16, 2024

eliteprox left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ad-astra-video Aug 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ad-astra-video commented Aug 17, 2024 • edited Loading

rickstaa commented Aug 19, 2024

ad-astra-video commented Aug 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rickstaa left a comment

Choose a reason for hiding this comment

ad-astra-video commented Aug 3, 2024 •

edited

Loading

eliteprox left a comment •

edited

Loading

ad-astra-video Aug 17, 2024 •

edited

Loading

ad-astra-video commented Aug 17, 2024 •

edited

Loading