-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.html
102 lines (102 loc) · 6.85 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
<title>Twitter Web Scraper</title>
<style type="text/css">
img {
max-width: 1000px;
height: auto;
}
</style>
</head>
<body>
<form action="https://www.paypal.com/cgi-bin/webscr" method="post" target="_top">
<input type="hidden" name="cmd" value="_donations" />
<input type="hidden" name="business" value="N8B796NE3Z6EQ" />
<input type="hidden" name="currency_code" value="USD" />
<input type="image" src="https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif" border="0" name="submit" title="PayPal - The safer, easier way to pay online!" alt="Donate with PayPal button" />
<img alt="" border="0" src="https://www.paypal.com/en_US/i/scr/pixel.gif" width="1" height="1" />
</form>
<pre style="color: yellow; background-color: black;">
_______ _ _ _ __ __ _ _____
|__ __| (_) | | | \ \ / / | | / ____|
| |_ ___| |_| |_ ___ _ __ \ \ /\ / /__| |__ | (___ ___ _ __ __ _ _ __ ___ _ __
| \ \ /\ / / | __| __/ _ \ '__| \ \/ \/ / _ \ '_ \ \___ \ / __| '__/ _' | '_ \ / _ \ '__|
| |\ V V /| | |_| || __/ | \ /\ / __/ |_) | ____) | (__| | | (_| | |_) | __/ |
|_| \_/\_/ |_|\__|\__\___|_| \/ \/ \___|_.__/ |_____/ \___|_| \__,_| .__/ \___|_|
| |
|_|
</pre>
<h2 style="text-decoration: underline;">Requirements</h2>
<ul>
<li>
<p><strong>Microsoft Windows</strong> operating system</p>
<p style="font-style: italic"> (tested on <strong>Windows 10</strong> Version 1909 OS build 18363.720 and <strong>Windows Server 2016</strong> Version 1607 OS build 14393.3564)</p>
</li>
<li>
<p><strong>Windows PowerShell</strong>, minimum version 3.0</p>
<p style="font-style: italic"> (tested PowerShell versions <strong>3.0</strong>, <strong>5.1.14393.3471</strong>, and <strong>5.1.18362.628</strong>. I have not tested PowerShell Core.)</p>
</li>
<li>
<p>A Twitter account (<strong>username</strong> and <strong>password</strong>)</p>
</li>
<li>
<p>A Twitter page that you would like to srape the media from</p>
</li>
</ul>
<h2 style="text-decoration: underline;">Installation</h2>
<ul>
<li>
<p><strong>Launch PowerShell and run the code below:</strong></p>
<p style="font-family: consolas"><pre><code>mkdir C:\TEMP\BIN -ea 0
cd C:\TEMP\BIN
git clone https://github.com/nstevens1040/PSTwitter-Media-Scraper.git
cd PSTwitter-Media-Scraper
. .\PSTwitter-Media-Scraper.ps1</code></pre></p>
</li>
<li>
<p><strong>You can also launch PowerShell and run:</strong></p>
<p style="font-family: consolas"><pre><code>iex (irm "https://raw.githubusercontent.com/nstevens1040/PSTwitter-Media-Scraper/master/PSTwitter-Media-Scraper.ps1")</code></pre></p>
</li>
</ul>
<p> Either one will make the function<strong> Scrape-TWPage</strong> available in your current PowerShell session.</p>
<h2 style="text-decoration: underline;">Environment Variables</h2>
<p> The first time you run<strong> Scrape-TWPage</strong> you will be asked to set two environment variables:</p>
<ol>
<li>
<p><strong> TWDOWNLOAD</strong>: Points to a root folder to store all of your download images and videos.</p>
<p><img alt="ScreenShot" src="https://github.com/nstevens1040/PSTwitter-Media-Scraper/raw/master/.gitignore/Capture1.PNG" /></p>
</li>
<li>
<p><strong> EXWEBREQ</strong>: Points to a root folder to store any .NET assemblies necessary to use <a href="https://github.com/nstevens1040/Execute-WebRequest"> Execute-WebRequest</a>.</p>
<p><img alt="ScreenShot" src="https://github.com/nstevens1040/PSTwitter-Media-Scraper/raw/master/.gitignore/Capture2.PNG" /></p>
</li>
</ol>
<h2 style="text-decoration: underline;">Usage</h2>
<ul>
<li>
<p><strong>Scrape-TWPage</strong> takes one argument called<strong> -TARGET_URI</strong>. This is the Uri that takes you to the Twitter page that you would like to scrape the media off of.</p>
</li>
<li>
<p><strong>SYNTAX:</strong> Scrape-TWPage [[-TARGET_URI] <string>] [<CommonParameters>]</p>
</li>
<li>
<p>A Windows credential dialog appears, asking you to enter the<strong> username</strong> and<strong> password</strong> that you use to login to Twitter.</p>
<p style="color:red"><strong>Your username, password, bearer tokens, and csrf tokens are protected with the <a href="https://docs.microsoft.com/en-us/dotnet/api/system.security.cryptography.protecteddata?view=netframework-4.8" target="_blank">System.Security.Cryptography.ProtectedData</a> .NET Framework class which provides access to Microsoft's Data Protection API.</strong></p>
</li>
</ul>
<p> At this point, you are finished entering the information necessary to proceed. The script's output will keep you updated on it's progress.</p>
<p> The steps above are illustrated in the video below.</p>
<p>
<img alt="ScreenShot" src="https://github.com/nstevens1040/PSTwitter-Media-Scraper/raw/master/.gitignore/PSTwitterScraperUsage.gif" />
</p>
<form action="https://www.paypal.com/cgi-bin/webscr" method="post" target="_top">
<input type="hidden" name="cmd" value="_donations" />
<input type="hidden" name="business" value="N8B796NE3Z6EQ" />
<input type="hidden" name="currency_code" value="USD" />
<input type="image" src="https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif" border="0" name="submit" title="PayPal - The safer, easier way to pay online!" alt="Donate with PayPal button" />
<img alt="" border="0" src="https://www.paypal.com/en_US/i/scr/pixel.gif" width="1" height="1" />
</form>
</body>
</html>