1.2.0 release

CHANGELOG.md

@@ -110,7 +110,7 @@ Functional changes
 - added Auto-Delete Seeded Torrents to automatically delete completely downloaded and seeded torrents from Transmission, leaving behind just the torrent's contents
 - updated Transmission icon with latest official design
 - fixed but temporarily removed Episodes Only toggle from config panel (entire concept of Episodes needs to be reworked now that print media can be faved)
-- removed all NMT and Mac OS X support (only supported OS is currently Debian 7.x)
+- removed all NMT and Mac OS X support
 - fixed feed section header and filter button match counts so that they obey the selected filter
 - added horizontal scrolling capability to History panel for long titles
 
@@ -660,6 +660,8 @@ Functional changes
 - Favorite > Also Save Dir now defaults to global Also Save Torrent Files Dir if left blank or if path is not writable ("Default" keyword no longer works)
 - Favorite > Download Dir now defaults to global Download Dir if left blank ("Default" keyword no longer works)
 - adjusted Configure > Feeds so that rows don't wrap when vertical scrollbar shows
+- bulk favorite importer now recognizes one field as both Name and Filter
+- fixed detection of Roman numeral seasons
 
 Code changes
 
@@ -684,15 +686,33 @@ Code changes
 - fixed bug in writeToLog() where ERROR wrote every level
 - removed $SERVER global from twxa_cli.php
 - fixed bug where FULL seasons are not treated as batches
+- feed type is correctly saved to config file
 
-Next Version
+1.2.0
 
 Functional Changes
 
-- removed Default setting keyword from Favorite > Also Save Dir and Favorite > Download Dir (will allow saving in a directory called Default)
+- CURL now passes through the user's User-Agent header value
+- due to systemd's PrivateTmp security feature in Ubuntu 18.04:
+  - moved /tmp/twxalog to /var/log/twxalog
+  - moved /tmp/.Transmission-Session-Id to /var/lib/torrentwatch-xa/dl_cache/.Transmission-Session-Id
+  - updated documentation to reflect new twxalog path
+- fixed detection of Roman-numeral season
+- changed client_add_torrent() to use torrent hash from URL if DDoS blockers like CloudFlare drop the connection
+
+Code Changes
+
+- renamed get_tr_sessionIdFile() to getTransmissionSessionIdFile()
+- moved getTransmissionSessionIdFile() from config.php to twxa_config_lib.php
+- added explicit int casts to validateYYYYMMDD()
+
+Next Version
+
+Functional Changes
 
 IN PROGRESS
 
+- handle gzipped torrent file (using gzuncompress() if file contents are returned directly or gzopen() and gzread() if file is downloaded)
 - add PHP prerequisite check to twxa_cli.php
 - fix rare bug where button bar stays visible when multiple items are trashed from Transmission list
 - fix vertical alignment of title line in Transmission filter on iPhone (first line of text sits too low and is too close to the progress bar)

INSTALL.md

@@ -50,11 +50,11 @@ torrentwatch-xa handles two passwords: one for Transmission and one for SMTP. If
 Installation Script
 ===============
 
-There is a rudimentary install/upgrade script called `install_twxa.sh` meant for Debian 8.x/9.x (but not Debian 7.x) and Ubuntu 14.04/16.04/18.04. It does work with the as-yet unsupported Fedora Server 25.
+There is a rudimentary install/upgrade script called `install_twxa.sh` compatible with Debian 8.x/9.x (but not Debian 7.x), Ubuntu 14.04/16.04/18.04, and Fedora Server 25.
 
 The install/upgrade script will remove an existing installation of torrentwatch-xa and only performs the copy and chown steps to put a fresh install in place. It does not install any prerequisite packages for you, nor does it configure or start/restart the Apache2 webserver. See Manual Installation below for those steps.
 
-Be aware that the script contains `rm -fr` commands, which are potentially dangerous. **Use install_twxa.sh at your own risk!** I will gradually improve the script over time until it essentially does every installation step, at which point it would probably be easiest to provide a .deb installation package.
+Be aware that the script contains `rm -fr` commands, which are potentially dangerous. **Use install_twxa.sh at your own risk!** I will gradually improve the script over time until it essentially does every installation step, at which point it would probably be best to provide a .deb installation package.
 
 To use the script, make sure you have sudo privileges or are running as root, then:
 
@@ -65,7 +65,7 @@ Then, if you are upgrading and want to keep your previous config:
 
 - `./install_twxa.sh --keep-config`
 
-Or for fresh installs or where you want to discard your config:
+For fresh installs or when you want to discard your config:
 
 - `./install_twxa.sh`
 
@@ -90,6 +90,9 @@ Manual Installation
 - Copy/move the folders and their contents to their intended locations:
   - `sudo mv ./torrentwatch-xa/var/www/html/torrentwatch-xa /var/www/html`
   - `sudo mv ./torrentwatch-xa/var/lib/torrentwatch-xa /var/lib`
+- Create the log file:
+  - `sudo touch /var/log/twxalog`
+  - `sudo chown www-data:www-data /var/log/twxalog`
 - Allow apache2 to write to the cache folders.
   - `sudo chown -R www-data:www-data /var/lib/torrentwatch-xa/*_cache`
 - Set up the cron job by copying the cron job script torrentwatch-xa-cron to /etc/cron.d with proper permissions for it to run.
@@ -124,6 +127,9 @@ __RedHat-derived distros are not officially supported at this time__ though the
 - Copy/move the folders and their contents to their intended locations:
   - `mv ./torrentwatch-xa/var/www/html/torrentwatch-xa /var/www/html`
   - `mv ./torrentwatch-xa/var/lib/torrentwatch-xa /var/lib`
+- Create the log file:
+  - `sudo touch /var/log/twxalog`
+  - `sudo chown apache:apache /var/log/twxalog`
 - Allow httpd to write to the cache folders.
   - `sudo chown -R apache:apache /var/lib/torrentwatch-xa/*_cache`
 - Set up the cron job by copying the cron job script torrentwatch-xa-cron to /etc/cron.d with proper permissions for it to run.

README.md

@@ -28,23 +28,11 @@ Common setups:
 Status
 ===============
 
-__NOTE:__ In Ubuntu 18.04 (not yet supported), there is a new systemd security feature called PrivateTmp preventing Apache2 from writing to /tmp/twxalog; instead, a temporary, private, virtual /tmp folder is created for Apache2 to write to. Meanwhile, the cron job continues to write to /tmp/twxalog. This results in there being two twxalog files on the system. __PrivateTmp does not affect the normal function of torrentwatch-xa: it will still download items properly. However, if you don't want dual log files, you can opt to disable PrivateTmp for Apache2.__ In 1.2.0, the files placed in /tmp will be moved to other locations so that PrivateTmp does not have to be disabled.
+I've posted 1.2.0 with the changes listed in [CHANGELOG.md](CHANGELOG.md).
 
-I've posted 1.1.0 with the changes listed in [CHANGELOG.md](CHANGELOG.md).
+1.2.0 can detect a torrent hash in the URL and use it if it cannot retrieve a torrent file. This ability can get around DDoS blockers like CloudFlare if the URL happens to contain the hash. (Thanks to efelon on GitHub for reporting this issue.)
 
-__twxacli.php has been renamed to twxa_cli.php to conform to the file naming convention; please be aware that the cron job file will need to be updated to call twxa_cli.php__
-
-1.1.0 fixed some leftover bugs in changing install paths by moving the hidden settings Cache Dir and History from torrentwatch-xa.config to twxa_config_lib.php.
-
-A very bad but very rare bug where the Configure settings were wiped and replaced with some default settings was finally fixed in 1.1.0. The bug was carried over from TorrentWatch-X 0.8.9 but was made worse by the conversion of the config file to JSON. Fixing it required a major rewrite of readjSONConfigFile() to change the way torrentwatch-xa.config and torrentwatch-xa-config.cache are handled.
-
-This config file bug had to be fixed in order to add a new, experimental feature: a command-line bulk Favorites importer, twxa_fav_import.php, now in alpha. You can now import a plain-text TSV (tab-separated values) file directly into your Favorites. Please read the usage notes for instructions.
-
-The formerly hidden setting debugLevel was renamed to Log Level and exposed via Configure > Interface > Log Level
-
-There's a new configuration option: Configure > Interface > Check for Updates
-
-1.1.0 has improved checks for important files or directories that do not exist or have incorrect permissions.
+To obviate Ubuntu 18.04's PrivateTmp feature, the twxalog file has been moved to /var/log/txwalog. If you use logrotate to rotate twxalog, remember to use this new path.
 
 #### Still in Alpha
 

TODO.md

@@ -54,6 +54,7 @@ All other files have functions that need improvement or rewrites or validation.
 
 ## Improvements
 
+- implement Ignore Batches feature per Favorite as well as globally
 - Clear Cache dialog closes automatically after any button is pressed
 - remove uninitialized variables in twxa_atomparser.php
 - store item version numbers in Favorite Last Downloaded field

TROUBLESHOOTING.md

@@ -7,7 +7,7 @@ There are four places to check for error messages:
 
 - a pop-up in the web browser (typically configuration errors such as permissions problems or unreachable paths)
 - web server error log: /var/log/apache2/error.log (all PHP errors, warnings, and notifications)
-- torrentwatch-xa error log: /tmp/twxalog (all errors in torrentwatch-xa logic such as failures to process feeds or auto-download items)
+- torrentwatch-xa error log: /var/log/twxalog (all errors in torrentwatch-xa logic such as failures to process feeds or auto-download items)
 - web browser Javascript console (Javascript errors only, typically minor errors in torrentwatch-xa's web UI)
 
 #### If you change the installation path(s)
@@ -28,7 +28,7 @@ Some feeds can't be added for various reasons including:
 
 - rejection of SSL certificate chain
 - feed format is not recognized as Atom or RSS
-- can't handle HTTP redirects
+- can't handle HTTP redirects or DDoS blockers like CloudFlare
 
 Please note that I have not been able to personally test torrentwatch-xa's handling of Atom feeds due to their extreme rarity.
 
@@ -67,7 +67,7 @@ First, please review the section of [INSTALL.md](INSTALL.md) called **Use the Fa
 
 Second, check the Favorite's Quality filter and make sure it's not too restrictive, then make sure there are no typos in any of the Favorite's fields.
 
-Third, check torrentwatch-xa's log (typically /tmp/twxalog) for errors.
+Third, check torrentwatch-xa's log (typically /var/log/twxalog) for errors.
 
 If you have followed the instructions correctly and are still having trouble, turn on Configure > Interface > Show Item Debug Info and refresh the browser so that you can see the show_title that must be matched by your Favorite Filter. You will likely find a typo in your Favorite's Filter that needs correcting.
 
@@ -113,7 +113,7 @@ You should see entries like these:
 
 Otherwise you will likely see errors with short instructions on how to fix the problem(s).
 
-It may also be necessary to check torrentwatch-xa's log file (typically /tmp/twxalog) for other errors preventing the auto-download.
+It may also be necessary to check torrentwatch-xa's log file (typically /var/log/twxalog) for other errors preventing the auto-download.
 
 #### "Invalid or corrupt torrent file"
 

USAGE.md

@@ -39,7 +39,7 @@ To run a shell script, check Enable Script, provide the full path to a single sh
 
 To use the built-in SMTP notifications, check SMTP Notifications and fill in the From Email and To Email fields and all the SMTP fields. SMTP Port defaults to 25 if left blank. From Email defaults to To Email if left blank or it is invalid. 
 
-For various in-depth reasons I am not providing any means of testing the SMTP settings at this time. You can trigger a real email notification by initiating a Favorite download (by cron job) and checking the log file at /tmp/twxalog for errors.
+No means of testing the SMTP settings are provided at this time. You can trigger a real email notification by initiating a Favorite download (by cron job) and checking the log file at /var/log/twxalog for errors.
 
 torrentwatch-xa uses PHPMailer 5.2.23 to send emails, so you may need to refer to PHPMailer documentation for help in understanding any SMTP error messages that appear.
 
@@ -76,19 +76,19 @@ The Episodes filter currently in each Favorite is still the old TorrentWatch-X f
 
 The Favorites fields behave differently in RegEx Matching Style than in Simple or Glob in that PCRE Unicode regular expressions are used in the Filter, Not, and Qualities fields in RegEx mode.
 
-### Authentication for private RSS Feeds
+### Authentication for Private RSS Feeds
 
 See the section "Only Public Torrent RSS or Atom Feeds Are Supported" in the **Design Decisions Explained** section below for more details.
 
 ### Auto-Del Seeded Torrents
 
-As of 0.5.0, Auto-Del Seeded Torrents has been fully implemented such that when enabled, either the web UI or twxa_cli.php (run by the cron job) will automatically delete completely-downloaded, fully-seeded torrents from Transmission without trashing the torrent's contents. Auto-Del Seeded Torrents is also smart enough not to delete any torrents that are not found in the download cache, preventing it from deleting torrents that were added to Transmission via other means.
+When enabled, either the web UI or twxa_cli.php (run by the cron job) will automatically delete completely-downloaded, fully-seeded torrents from Transmission without trashing the torrent's contents. Auto-Del Seeded Torrents is also smart enough not to delete any torrents that are not found in the download cache, preventing it from deleting torrents that were added to Transmission via other means.
 
-As of 0.6.0, the web UI is fully synchronized with Transmission so that items auto-deleted by twxa_cli.php will be removed from the web UI correctly without requiring a browser refresh.
+The web UI is fully synchronized with Transmission so that items auto-deleted by twxa_cli.php will be removed from the web UI correctly without requiring a browser refresh.
 
-### Saving Magnet Links as Files
+### Magnet Links Saved as Files
 
-As of 0.6.1, the features that save torrent files to the filesystem can now save magnet links as files. This seems counterintuitive, since the point of magnet links is to avoid having to deal with downloading, storing, and hosting torrent files. However, there is no other way to record magnet links for later retrieval except by writing them to a file, so that's what torrentwatch-xa does. Luckily, there are third-party tools that easily convert magnet links stored in text files to torrent files, if you prefer the torrent file over the magnet link.
+If Also Save Torrent Files is enabled and torrentwatch-xa retrieves a magnet link instead of a .torrent file, it will save the magnet link in a file. There are third-party tools that easily convert magnet links stored in text files to torrent files, if you prefer the torrent file over the magnet link.
 
 The ability to save magnet links was added to deal with the increasingly-common feeds that have only magnet links and no links to torrent files.
 
@@ -116,11 +116,9 @@ Then, at the command line, run:
 
 `sudo /usr/bin/php /var/www/html/torrentwatch-xa/twxa_fav_import.php <path to TSV>`
 
-You can watch the web server's log file (default: /var/log/apache2/error.log) for PHP errors and torrentwatch-xa's log (default: /tmp/twxalog) for import errors.
-
+You can watch the web server's log file (default: /var/log/apache2/error.log) for PHP errors and torrentwatch-xa's log (default: /var/log/twxalog) for import errors.
 If there are no errors, go ahead and open torrentwatch-xa in the browser and make sure the new Favorites are imported.
 
-
 If the TSV file confuses PHP's fgetcsv() function, there is a good possibility you will corrupt your config file. If you have to put the backup file back, do this:
 
 Close the browser if you have torrentwatch-xa's web UI open.
@@ -163,4 +161,4 @@ If one starts an item downloading from a feed list, and that item is bumped off
 
 ### Watch Dir
 
-As it turns out, the Watch Dir functionality has not actually worked since TorrentWatch-X 0.8.9 due to the design of find_torrent_link(). Because transmission-daemon already has a much faster watch directory capability built-in, Watch Dir has been removed from torrentwatch-xa as of 0.5.0 rather than repaired. To enable the watch directory in transmission-daemon, use `watch-dir` and `watch-dir-enabled` in `settings.json`.
+transmission-daemon provides a watch directory feature. To enable it, use `watch-dir` and `watch-dir-enabled` in `settings.json`.

install_twxa.sh

@@ -93,6 +93,16 @@ sudo cp -R var/www/html/torrentwatch-xa /var/www/html
 sudo cp etc/cron.d/torrentwatch-xa-cron /etc/cron.d
 sudo chown root:root /etc/cron.d/torrentwatch-xa-cron
 
+# create the log file
+echo "Creating log file..."
+sudo touch /var/log/twxalog
+sudo chown www-data:www-data /var/log/twxalog
+if [ $? -ne 0 ]
+then
+    # try to chown the cache directories using Fedora default Apache user and group apache
+  sudo chown apache:apache /var/log/twxalog
+fi
+
 # copy in the old config file
 echo "Copying the old config file back into the new installation..."
 if [[ $KEEPCONFIG == 1 ]]

var/lib/torrentwatch-xa/lib/twxa_config_lib.php

@@ -23,6 +23,10 @@ function getDownloadHistoryFile() {
     return get_baseDir() . "/dl_cache/dl_history";
 }
 
+function getTransmissionSessionIdFile() {
+    return getDownloadCacheDir() . "/.Transmission-Session-Id";
+}
+
 function getTransmissionrPCPath() {
     return "/transmission/rpc"; // do not change this without good reason
 }

var/lib/torrentwatch-xa/lib/twxa_parse.php

@@ -74,10 +74,10 @@ function normalizeCodecs($ti, $seps = '\s\.\_') {
 }
 
 function validateYYYYMMDD($date) {
-    $YYYY = substr($date, 0, 4);
-    $MM = substr($date, 4, 2);
-    $DD = substr($date, 6, 2);
-    return checkdate($MM + 0, $DD + 0, $YYYY + 0);
+    $YYYY = (int)substr($date, 0, 4);
+    $MM = (int)substr($date, 4, 2);
+    $DD = (int)substr($date, 6, 2);
+    return checkdate($MM, $DD, $YYYY);
 }
 
 function simplifyTitle($ti) {

var/lib/torrentwatch-xa/lib/twxa_tools.php

@@ -88,13 +88,20 @@ function array_change_key_case_ext($array, $case = ARRAY_KEY_LOWERCASE) {
 }
 
 function getcURLDefaults(&$curlOptions) {
-    $curlOptions[CURLOPT_CONNECTTIMEOUT] = 15;
+    global $twxa_version;
+    $curlOptions[CURLOPT_CONNECTTIMEOUT] = 20; // was 15
     $curlOptions[CURLOPT_SSL_VERIFYPEER] = false;
     $curlOptions[CURLOPT_SSL_VERIFYHOST] = false;
     $curlOptions[CURLOPT_FOLLOWLOCATION] = true;
     $curlOptions[CURLOPT_UNRESTRICTED_AUTH] = true;
-    $curlOptions[CURLOPT_TIMEOUT] = 20;
+    $curlOptions[CURLOPT_TIMEOUT] = 30; // was 20
     $curlOptions[CURLOPT_RETURNTRANSFER] = true;
+    if (filter_input(INPUT_SERVER, "HTTP_USER_AGENT") == "") {
+        $curlOptions[CURLOPT_USERAGENT] = "torrentwatch-xa/$twxa_version[0] ($twxa_version[1])";
+        //$curlOptions[CURLOPT_USERAGENT] = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13';
+    } else {
+        $curlOptions[CURLOPT_USERAGENT] = filter_input(INPUT_SERVER, "HTTP_USER_AGENT");
+    }
     return($curlOptions);
 }
 
@@ -134,7 +141,7 @@ function writeToLog($string, $lvl = -1) {
           $debug_output = "<script type='text/javascript'>alert('$string');</script>"; //TODO append errors to some global that will be echoed to the HTML output buffer just once
           } */
         // write plain text to log file
-        if(file_put_contents(get_logFile(), date("c") . " $errLabel $string", FILE_APPEND) === false) {
+        if (file_put_contents(get_logFile(), date("c") . " $errLabel $string", FILE_APPEND) === false) {
             //TODO failed to write, send error to HTML
         }
     }
@@ -272,4 +279,4 @@ function parseURLForCookies($url) {
         $cookies = strtr(substr($cookies, 8), '&', ';');
         return array('url' => $url, 'cookies' => $cookies);
     }
-}
+}