Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Actor self-deletion. #230

Open
olitomlinson opened this issue Oct 3, 2019 · 6 comments
Open

Support Actor self-deletion. #230

olitomlinson opened this issue Oct 3, 2019 · 6 comments

Comments

@olitomlinson
Copy link

Is your feature request related to a problem? Please describe.

I'm currently designing a multi-tenant high-throughput system that will be creating approximately 50-150 million short-lived Actors per day depending on customer demand.

An Actor will perform its duty and come to a conclusion, publish its results to EventGrid, and then become obsolete. The Actor does not have a natural owner or supervisor that is in control of its lifetime.

In order to perform this deletion, (because there is no owner....) I must delegate the responsibility of the deletion to an external process, which can then invoke the Actor delete method.

This deletion process could be hosted as a simple and straight-forward Stateful Service, but this is one more service that I need to own and operate, which exists purely as a side-effect of the technology.

In order to keep the services in the domain focused on business value, I would prefer not to have this additional responsibility and cognitive load.

Describe the solution you'd like

I would like an Actor to be able to delete itself.

The Deletion API could be borrow stylistically from the IRemindable interface. Something like :

var _registration = RegisterDeletion(TimeSpan.FromMinutes(15)); //schedule the Actor for deletion in 15 minutes time.
var _registration = RegisterDeletion(0); //schedule the Actor for immediate deletion.
UnregisterDeletion(_registration); //Things have changed, I no longer want to delete the Actor

Describe alternatives you've considered

As described above, I would have to create another service that has the responsibility of performing the deletion. This could be done internally in the cluster by introducing a new Stateful Service to own the process, but would utilise more precious cluster resources, such as :

  • Potentially millions of RPC network calls into the Stateful Service for scheduling deletion.
  • Potentially millions of RPC network calls into the Actor for actual deletion.
  • Increased memory pressure for the Reliable Collections to store deletion requests.
  • Increased CPU for processing the deletion requests.
  • Increased disk utilisation for data redundancy.
  • Another point of failure.

Additional context

What would happen when deletion is not honoured successfully and internal retries have been exhausted? There is no supervisor or owner to notify. Would it raise a Health Event? In the event of failure, something would need to be logged somewhere as there could potentially be orphaned Actor metadata sitting around in memory forever which must be cleaned-up somehow.

@olitomlinson olitomlinson changed the title [Enhancement] Support Actor self-deletion. Support Actor self-deletion. Oct 16, 2019
@AlkisFortuneFish
Copy link

One workaround until (and if) this is supported is to have the actor fire off deletion with a Task.Run(), without awaiting it and immediately returning.

@BuddhaBuddy1
Copy link

One workaround until (and if) this is supported is to have the actor fire off deletion with a Task.Run(), without awaiting it and immediately returning.

If there were a problem with the task, it would not be deleted. There needs to be a reliable deletion method.

It seems very strange to me that you can't just mark it for deletion, implement OnDeleteAsync(), and the GC deletes it the next sweep.

@AlkisFortuneFish
Copy link

If there were a problem with the task, it would not be deleted. There needs to be a reliable deletion method.

Indeed. In newer code I achieve that at the service level, an actor registers itself for deletion by calling code in the actor service, which persists the request and then deletes it afterwards.

It seems very strange to me that you can't just mark it for deletion, implement OnDeleteAsync(), and the GC deletes it the next sweep.

Totally agreed.

@BuddhaBuddy1
Copy link

I figured I would have to do something like that. It is just silly that I have to create a new Stateful Service just to have them be able to reliably delete themselves (I'm invoking them from a Stateless Service).

@AlkisFortuneFish
Copy link

AlkisFortuneFish commented Mar 1, 2021

You can do it directly on the ActorService itself, with RunAsync periodically running the deletions, although the lack of direct access to the underlying state provider means that you have to persist state via IActorStateProvider and a sacrificial actorId, which feels quite hacky. Unless your actors don't use persisted state of course. All in all, I haven't found any truly nice solutions to this.

@olitomlinson
Copy link
Author

Stateful Services programming models getting no love for a few years now. Very disappointing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants