Merge branch 'feature/forced-tick-changes' into develop

CHANGELOG.md

@@ -10,6 +10,7 @@
 * An activity indicator now briefly flashes green on the overlay when BGS-Tally logs BGS or TW activity.
 * Thargoid War reactivation (settlement reboot) missions are now tracked: `🛠️ (missions)`, both for the station issuing the mission and for the system where the settlement was reactivated.
 * Added a new setting to allow you to switch off reporting for new systems you visit. This is for CMDRs who regularly only want to report a subset of their work - it means you don't have to switch off a load of systems, you can just switch on the few you need.
+* Forced ticks are now labelled clearly, including in Discord posts.
 
 ### Changes:
 
@@ -20,12 +21,19 @@
 
 * Some Orthrus kills were not being tallied because the bond value logged was 40m instead of the previous 25m. We can only detect the type of Thargoid via the bond value logged by the game, so BGS-Tally will now tally an Orthrus for both kill values.
 * Trade purchase, sale and profit was not being logged if you previously disembarked from your ship on foot, took a taxi or dropship somewhere, returned to your ship and then traded.
-* If you did a Force Tick but then an automatic tick was detected that had an earlier tick time, this could cause BGS-Tally to get confused, post more than once to Discord and if you also restart EDMC you could lose activity. Now in this situation, your current activity inherits the older incoming tick time.
+* Forcing a tick (via the settings panel), though still not advised unless automatic tick detection has **definitely** missed a tick, should now be much more reliable:
+    * It would cause your previously logged activity for the current tick to be lost and replaced by activity after the forced tick. Now, a 'proper' new tick is created so your earlier activity should be kept and available in the previous tick.
+    * If an automatic tick arrived with an earlier tick time than your forced tick, this could cause BGS-Tally to get confused. We now ignore any incoming ticks that have an older tick time than your forced tick.
+    * Forced ticks are now handled more elegantly when sending data via the BGS-Tally API, as we generate a fake `tickid` for the forced tick.
 
 ### API Changes ([v1.2](https://studio-ws.apicur.io/sharing/281a84ad-dca9-42da-a08b-84e4b9af1b7e)):
 
 * `/activities` endpoint: Thargoid War reactivation missions now included in `systems/[system]/factions/[faction]/stations/[station]/twreactivate`
 * `/activities` endpoint: Thargoid War number of settlements reactivated now included in `systems/[system]/twreactivate`
+* `/activities` endpoint: `Activity` now has a `ticktime` timestamp in addition to `tickid`.
+* `/activities` endpoint: When the user forces a tick, a new `tickid` is generated by BGS-Tally that conforms to the 24-character elitebgs.app tickid standard but starts with six zeroes '000000' to distinguish it as a forced tick.
+* `/events` endpoint: `Event` now has a `ticktime` timestamp in addition to `tickid`.
+* `/events` endpoint: When the user forces a tick, a new `tickid` is generated by BGS-Tally that conforms to the 24-character elitebgs.app tickid standard but starts with six zeroes '000000' to distinguish it as a forced tick.
 
 
 ## v3.1.1 - 2023-08-23

bgstally/activity.py

@@ -3,7 +3,7 @@
 from datetime import datetime, timedelta
 from typing import Dict
 
-from bgstally.constants import CheckStates, DiscordActivity
+from bgstally.constants import CheckStates, DiscordActivity, FILE_SUFFIX
 from bgstally.debug import Debug
 from bgstally.missionlog import MissionLog
 from bgstally.state import State
@@ -12,6 +12,7 @@
 from thirdparty.colors import *
 
 DATETIME_FORMAT_ACTIVITY = "%Y-%m-%dT%H:%M:%S.%fZ"
+DATETIME_FORMAT_TITLE = "%Y-%m-%d %H:%M:%S"
 STATES_WAR = ['War', 'CivilWar']
 STATES_ELECTION = ['Election']
 
@@ -94,14 +95,15 @@ def __init__(self, bgstally, tick: Tick = None, discord_bgs_messageid: str = Non
         self.bgstally = bgstally
         if tick == None: tick = Tick(self.bgstally)
 
-        # Stored data. Remember to modify __deepcopy__() if these are changed or new data added.
-        self.tick_id = tick.tick_id
-        self.tick_time = tick.tick_time
-        self.discord_bgs_messageid = discord_bgs_messageid
-        self.discord_tw_messageid = None
-        self.discord_notes = ""
+        # Stored data. Remember to modify __deepcopy__(), _as_dict() and _from_dict() if these are changed or new data added.
+        self.tick_id: str = tick.tick_id
+        self.tick_time: datetime = tick.tick_time
+        self.tick_forced: bool = False
+        self.discord_bgs_messageid: str = discord_bgs_messageid
+        self.discord_tw_messageid: str = None
+        self.discord_notes: str = ""
         self.dirty: bool = False
-        self.systems = {}
+        self.systems: dict = {}
 
 
     def load_legacy_data(self, filepath: str):
@@ -149,6 +151,23 @@ def save(self, filepath: str):
             self.dirty = False
 
 
+    def get_filename(self) -> str:
+        """
+        Return the filename for this Activity
+        """
+        return self.tick_id + FILE_SUFFIX
+
+
+    def get_title(self) -> str:
+        """
+        Get the title for this activity
+        """
+        if self.tick_forced:
+            return f"{str(self.tick_time.strftime(DATETIME_FORMAT_TITLE))} (forced)"
+        else:
+            return f"{str(self.tick_time.strftime(DATETIME_FORMAT_TITLE))} (game)"
+
+
     def get_ordered_systems(self):
         """
         Get an ordered list of the systems we are tracking, with the current system first, followed by those with activity, and finally those without
@@ -1217,6 +1236,7 @@ def _as_dict(self):
         return {
             'tickid': self.tick_id,
             'ticktime': self.tick_time.strftime(DATETIME_FORMAT_ACTIVITY),
+            'tickforced': self.tick_forced,
             'discordmessageid': self.discord_bgs_messageid,
             'discordtwmessageid': self.discord_tw_messageid,
             'discordnotes': self.discord_notes,
@@ -1229,6 +1249,7 @@ def _from_dict(self, dict: Dict):
         """
         self.tick_id = dict.get('tickid')
         self.tick_time = datetime.strptime(dict.get('ticktime'), DATETIME_FORMAT_ACTIVITY)
+        self.tick_forced = dict.get('tickforced', False)
         self.discord_bgs_messageid = dict.get('discordmessageid')
         self.discord_tw_messageid = dict.get('discordtwmessageid')
         self.discord_notes = dict.get('discordnotes')
@@ -1273,6 +1294,7 @@ def __deepcopy__(self, memo):
         setattr(result, 'bgstally', self.bgstally)
         setattr(result, 'tick_id', self.tick_id)
         setattr(result, 'tick_time', self.tick_time)
+        setattr(result, 'tick_forced', self.tick_forced)
         setattr(result, 'discord_bgs_messageid', self.discord_bgs_messageid)
         setattr(result, 'discord_tw_messageid', self.discord_tw_messageid)
         setattr(result, 'discord_notes', self.discord_notes)

bgstally/activitymanager.py

@@ -4,14 +4,14 @@
 from config import config
 
 from bgstally.activity import Activity
+from bgstally.constants import FILE_SUFFIX
 from bgstally.debug import Debug
 from bgstally.tick import Tick
 
 FILE_LEGACY_CURRENTDATA = "Today Data.txt"
 FILE_LEGACY_PREVIOUSDATA = "Yesterday Data.txt"
 FOLDER_ACTIVITYDATA = "activitydata"
 FOLDER_ACTIVITYDATA_ARCHIVE = "archive"
-FILE_SUFFIX = ".json"
 KEEP_CURRENT_ACTIVITIES = 20
 
 
@@ -43,7 +43,7 @@ def save(self):
         """
         for activity in self.activity_data:
             if activity.tick_id is None: continue
-            activity.save(path.join(self.bgstally.plugin_dir, FOLDER_ACTIVITYDATA, activity.tick_id + FILE_SUFFIX))
+            activity.save(path.join(self.bgstally.plugin_dir, FOLDER_ACTIVITYDATA, activity.get_filename()))
 
 
     def get_current_activity(self):
@@ -60,21 +60,22 @@ def get_previous_activities(self):
         return self.activity_data[1:]
 
 
-    def new_tick(self, tick: Tick):
+    def new_tick(self, tick: Tick, forced: bool) -> bool:
         """
-        New tick detected, duplicate the current Activity object or update the current tick time.
+        New tick detected, duplicate the current Activity object or ignore if it's older than current tick.
         """
 
         if tick.tick_time < self.current_activity.tick_time:
-            # An inbound tick is older than the current tick. The only valid situation for this is if the user previously
-            # did a Force Tick and an earlier tick was later detected. Just set the current tick time to the received tick time.
-            self.current_activity.tick_time = tick.tick_time
+            # An inbound tick is older than the current tick. The only valid situation for this is if the user has done a Force Tick
+            # but an elitebgs.app tick was then detected with an earlier timestamp. Ignore the tick in this situation.
+            return False
         else:
             # An inbound tick is newer than the current tick. Create a new Activity object.
             # Note Activity uses a customised __deepcopy__ that only deep copies data, not class instances.
             new_activity:Activity = deepcopy(self.current_activity)
             new_activity.tick_id = tick.tick_id
             new_activity.tick_time = tick.tick_time
+            new_activity.tick_forced = forced
             new_activity.discord_bgs_messageid = None
             new_activity.discord_tw_messageid = None
             new_activity.discord_notes = ""
@@ -83,6 +84,8 @@ def new_tick(self, tick: Tick):
             self.activity_data.sort(reverse=True)
             self.current_activity = new_activity
 
+            return True
+
 
     def _load(self):
         """
@@ -121,7 +124,7 @@ def _convert_legacy_data(self, filepath: str, tick: Tick, discord_bgs_messageid:
 
         activity = Activity(self.bgstally, tick, discord_bgs_messageid)
         activity.load_legacy_data(filepath)
-        activity.save(path.join(self.bgstally.plugin_dir, FOLDER_ACTIVITYDATA, activity.tick_id + FILE_SUFFIX))
+        activity.save(path.join(self.bgstally.plugin_dir, FOLDER_ACTIVITYDATA, activity.get_filename()))
         self.activity_data.append(activity)
         if activity.tick_id == tick.tick_id: self.current_activity = activity
 
@@ -139,8 +142,8 @@ def _archive_old_activity(self):
 
         for activity in activity_to_archive:
             try:
-                rename(path.join(self.bgstally.plugin_dir, FOLDER_ACTIVITYDATA, activity.tick_id + FILE_SUFFIX),
-                       path.join(self.bgstally.plugin_dir, archive_filepath, activity.tick_id + FILE_SUFFIX))
+                rename(path.join(self.bgstally.plugin_dir, FOLDER_ACTIVITYDATA, activity.get_filename()),
+                       path.join(self.bgstally.plugin_dir, archive_filepath, activity.get_filename()))
             except FileExistsError: # Destination exists
                 Debug.logger.warning(f"Attempt to archive failed, destination file already exists")
                 continue

bgstally/api.py

@@ -13,7 +13,7 @@
 from bgstally.requestmanager import BGSTallyRequest
 from bgstally.utils import get_by_path
 
-API_VERSION = "1.1.0"
+API_VERSION = "1.2.0"
 
 ENDPOINT_ACTIVITIES = "activities" # Used as both the dict key and default path
 ENDPOINT_DISCOVERY = "discovery"   # Used as the path

bgstally/apimanager.py

@@ -84,6 +84,7 @@ def _build_api_activity(self, activity:Activity, cmdr:str):
         api_activity:dict = {
             'cmdr': cmdr,
             'tickid': activity.tick_id,
+            'ticktime': activity.tick_time.strftime(DATETIME_FORMAT_JOURNAL),
             'timestamp': datetime.utcnow().strftime(DATETIME_FORMAT_JOURNAL),
             'systems': []
         }
@@ -243,6 +244,7 @@ def _build_api_event(self, event:dict, activity:Activity, cmdr:str):
         # BGS-Tally specific global enhancements
         event['cmdr'] = cmdr
         event['tickid'] = activity.tick_id
+        event['ticktime']: activity.tick_time.strftime(DATETIME_FORMAT_JOURNAL)
         event['StationFaction'] = self.bgstally.state.station_faction
 
         # Other global enhancements

bgstally/bgstally.py

@@ -264,7 +264,7 @@ def new_tick(self, force: bool, uipolicy: UpdateUIPolicy):
         Start a new tick.
         """
         if force: self.tick.force_tick()
-        self.activity_manager.new_tick(self.tick)
+        if not self.activity_manager.new_tick(self.tick, force): return
 
         match uipolicy:
             case UpdateUIPolicy.IMMEDIATE:

bgstally/constants.py

@@ -66,6 +66,7 @@ class RequestMethod(Enum):
 
 
 DATETIME_FORMAT_JOURNAL = "%Y-%m-%dT%H:%M:%SZ"
+FILE_SUFFIX = ".json"
 FOLDER_ASSETS = "assets"
 FOLDER_DATA = "otherdata"
 FOLDER_BACKUPS = "backups"

bgstally/tick.py

@@ -3,6 +3,7 @@
 import plug
 import requests
 from config import config
+from secrets import token_hex
 
 from bgstally.debug import Debug
 
@@ -19,8 +20,8 @@ class Tick:
 
     def __init__(self, bgstally, load: bool = False):
         self.bgstally = bgstally
-        self.tick_id = TICKID_UNKNOWN
-        self.tick_time = (datetime.utcnow() - timedelta(days = 30)) # Default to a tick a month old
+        self.tick_id:str = TICKID_UNKNOWN
+        self.tick_time:datetime = (datetime.utcnow() - timedelta(days = 30)) # Default to a tick a month old
         if load: self.load()
 
 
@@ -37,11 +38,12 @@ def fetch_tick(self):
             return None
         else:
             tick = response.json()
+            tick_time:datetime = datetime.strptime(tick[0]['time'], DATETIME_FORMAT_ELITEBGS)
 
-            if self.tick_id != tick[0]['_id']:
-                # There is a new tick ID
+            if tick_time > self.tick_time:
+                # There is a newer tick
                 self.tick_id = tick[0]['_id']
-                self.tick_time = datetime.strptime(tick[0]['time'], DATETIME_FORMAT_ELITEBGS)
+                self.tick_time = tick_time
                 return True
 
         return False
@@ -51,8 +53,8 @@ def force_tick(self):
         """
         Force a new tick, user-initiated
         """
-        # Keep the same tick ID so we don't start another new tick on next launch,
-        # but update the time to show the user that something has happened
+        # Set the tick time to the current datetime and generate a new 24-digit tick id with six leading zeroes to signify a forced tick
+        self.tick_id = f"000000{token_hex(9)}"
         self.tick_time = datetime.now()
 
 

bgstally/ui.py

@@ -241,7 +241,7 @@ def _previous_ticks_popup(self):
         activities: List = self.bgstally.activity_manager.get_previous_activities()
 
         for activity in activities:
-            menu.add_command(label=activity.tick_time, command=partial(self._show_activity_window, activity))
+            menu.add_command(label=activity.get_title(), command=partial(self._show_activity_window, activity))
 
         try:
             menu.tk_popup(self.button_previous_ticks.winfo_rootx(), self.button_previous_ticks.winfo_rooty())

bgstally/windows/activity.py

@@ -7,13 +7,11 @@
 from bgstally.activity import STATES_WAR, Activity
 from bgstally.constants import FOLDER_ASSETS, FONT_HEADING, FONT_TEXT, CheckStates, CZs, DiscordActivity, DiscordChannel, DiscordPostStyle
 from bgstally.debug import Debug
-from bgstally.discord import DATETIME_FORMAT
 from bgstally.utils import human_format
 from bgstally.widgets import DiscordAnsiColorText, TextPlus
 from thirdparty.colors import *
 from thirdparty.ScrollableNotebook import ScrollableNotebook
 
-DATETIME_FORMAT_WINDOWTITLE = "%Y-%m-%d %H:%M:%S"
 LIMIT_TABS = 60
 
 
@@ -42,7 +40,7 @@ def _show(self, activity: Activity):
         Show our window
         """
         self.toplevel:tk.Toplevel = tk.Toplevel(self.ui.frame)
-        self.toplevel.title(f"{self.bgstally.plugin_name} - Activity After Tick at: {activity.tick_time.strftime(DATETIME_FORMAT_WINDOWTITLE)}")
+        self.toplevel.title(f"{self.bgstally.plugin_name} - Activity After Tick at: {activity.get_title()}")
 
         ContainerFrame = ttk.Frame(self.toplevel)
         ContainerFrame.pack(fill=tk.BOTH, expand=tk.YES)
@@ -303,21 +301,21 @@ def _post_to_discord(self, activity: Activity):
             if self.bgstally.state.DiscordActivity.get() == DiscordActivity.BGS:
                 # BGS Only - one post to BGS channel
                 discord_fields:Dict = activity.generate_discord_embed_fields(DiscordActivity.BGS)
-                self.bgstally.discord.post_embed(f"BGS Activity after tick: {activity.tick_time.strftime(DATETIME_FORMAT)}", description, discord_fields, activity.discord_bgs_messageid, DiscordChannel.BGS, self.discord_post_complete)
+                self.bgstally.discord.post_embed(f"BGS Activity after tick: {activity.get_title()}", description, discord_fields, activity.discord_bgs_messageid, DiscordChannel.BGS, self.discord_post_complete)
             elif self.bgstally.state.DiscordActivity.get() == DiscordActivity.THARGOIDWAR:
                 # TW Only - one post to TW channel
                 discord_fields:Dict = activity.generate_discord_embed_fields(DiscordActivity.THARGOIDWAR)
-                self.bgstally.discord.post_embed(f"TW Activity after tick: {activity.tick_time.strftime(DATETIME_FORMAT)}", description, discord_fields, activity.discord_tw_messageid, DiscordChannel.THARGOIDWAR, self.discord_post_complete)
+                self.bgstally.discord.post_embed(f"TW Activity after tick: {activity.get_title()}", description, discord_fields, activity.discord_tw_messageid, DiscordChannel.THARGOIDWAR, self.discord_post_complete)
             elif self.bgstally.discord.is_webhook_valid(DiscordChannel.THARGOIDWAR):
                 # Both, TW channel is available - two posts, one to each channel
                 discord_fields:Dict = activity.generate_discord_embed_fields(DiscordActivity.BGS)
-                self.bgstally.discord.post_embed(f"BGS Activity after tick: {activity.tick_time.strftime(DATETIME_FORMAT)}", description, discord_fields, activity.discord_bgs_messageid, DiscordChannel.BGS, self.discord_post_complete)
+                self.bgstally.discord.post_embed(f"BGS Activity after tick: {activity.get_title()}", description, discord_fields, activity.discord_bgs_messageid, DiscordChannel.BGS, self.discord_post_complete)
                 discord_fields:Dict = activity.generate_discord_embed_fields(DiscordActivity.THARGOIDWAR)
-                self.bgstally.discord.post_embed(f"TW Activity after tick: {activity.tick_time.strftime(DATETIME_FORMAT)}", description, discord_fields, activity.discord_tw_messageid, DiscordChannel.THARGOIDWAR, self.discord_post_complete)
+                self.bgstally.discord.post_embed(f"TW Activity after tick: {activity.get_title()}", description, discord_fields, activity.discord_tw_messageid, DiscordChannel.THARGOIDWAR, self.discord_post_complete)
             else:
                 # Both, TW channel is not available - one combined post to BGS channel
                 discord_fields:Dict = activity.generate_discord_embed_fields(DiscordActivity.BOTH)
-                self.bgstally.discord.post_embed(f"Activity after tick: {activity.tick_time.strftime(DATETIME_FORMAT)}", description, discord_fields, activity.discord_bgs_messageid, DiscordChannel.BGS, self.discord_post_complete)
+                self.bgstally.discord.post_embed(f"Activity after tick: {activity.get_title()}", description, discord_fields, activity.discord_bgs_messageid, DiscordChannel.BGS, self.discord_post_complete)
 
         activity.dirty = True # Because discord post ID has been changed