Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates the Github documentation for v1.0 release. #712

Open
wants to merge 59 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
03b7939
Adding section for UDF serialization
Niharikadutta Apr 20, 2020
4ef693d
removing guides from master
Niharikadutta Apr 20, 2020
81145ca
Merge latest from master
Niharikadutta May 6, 2020
e4b81af
merging latest from master
Niharikadutta May 7, 2020
4c32173
Merge remote-tracking branch 'upstream/master'
Niharikadutta Jun 2, 2020
4987a09
Merge remote-tracking branch 'upstream/master'
Niharikadutta Jun 14, 2020
ca9612e
Merge remote-tracking branch 'upstream/master'
Niharikadutta Jun 16, 2020
f581c86
Merge remote-tracking branch 'upstream/master'
Niharikadutta Jun 20, 2020
086b325
Merge remote-tracking branch 'upstream/master'
Niharikadutta Jun 23, 2020
2f72907
Merge remote-tracking branch 'upstream/master'
Niharikadutta Jul 25, 2020
6bab996
CountVectorizer
Jul 27, 2020
e2a566b
moving private methods to bottom
Jul 27, 2020
5f682a6
changing wrap method
Jul 28, 2020
31371db
setting min version required
Jul 31, 2020
60eb82f
undoing csproj change
Jul 31, 2020
ed36375
member doesnt need to be internal
Jul 31, 2020
c7baf72
too many lines
Jul 31, 2020
d13303c
removing whitespace change
Jul 31, 2020
f5b477c
removing whitespace change
Jul 31, 2020
73db52b
ionide
Jul 31, 2020
98f5e4d
Merge remote-tracking branch 'upstream/master'
Niharikadutta Aug 7, 2020
4c5d502
Merge remote-tracking branch 'upstream/master'
Niharikadutta Aug 10, 2020
a766146
Merge branch 'master' into ml/countvectorizer
GoEddie Aug 12, 2020
ad6bced
Merge branch 'ml/countvectorizer' of https://github.com/GoEddie/spark
Niharikadutta Aug 13, 2020
8e1685c
Revert "Merge branch 'master' into ml/countvectorizer"
Niharikadutta Aug 13, 2020
255515e
Revert "Merge branch 'ml/countvectorizer' of https://github.com/GoEdd…
Niharikadutta Aug 13, 2020
a44c882
Merge remote-tracking branch 'upstream/master'
Niharikadutta Aug 14, 2020
3c2c936
fixing merge errors
Niharikadutta Aug 14, 2020
88e834d
removing ionid
Niharikadutta Aug 20, 2020
59e7299
Merge remote-tracking branch 'upstream/master'
Niharikadutta Aug 20, 2020
a13de2d
Merge branch 'master' of github.com:Niharikadutta/spark
Niharikadutta Aug 21, 2020
13d0e4a
Merge remote-tracking branch 'upstream/master'
Niharikadutta Aug 24, 2020
595b141
Merge remote-tracking branch 'upstream/master'
Niharikadutta Aug 29, 2020
decfa48
Merge remote-tracking branch 'upstream/master'
Niharikadutta Sep 2, 2020
ce694ff
Merge remote-tracking branch 'upstream/master'
Niharikadutta Sep 8, 2020
8128ba0
Merge remote-tracking branch 'upstream/master'
Niharikadutta Sep 12, 2020
52f0a74
Merge remote-tracking branch 'upstream/master'
Niharikadutta Sep 19, 2020
6a89f01
Merge remote-tracking branch 'upstream/master'
Niharikadutta Sep 24, 2020
4b1de41
Merge remote-tracking branch 'upstream/master'
Niharikadutta Sep 25, 2020
929d8e2
Merge remote-tracking branch 'upstream/master'
Niharikadutta Sep 26, 2020
ffa0a4d
Merge remote-tracking branch 'upstream/master'
Niharikadutta Oct 2, 2020
2579faa
Merge remote-tracking branch 'upstream/master'
Niharikadutta Oct 5, 2020
39b3950
first draft
Niharikadutta Oct 5, 2020
2297add
Merge remote-tracking branch 'upstream/master'
Niharikadutta Oct 6, 2020
daade7a
Merge remote-tracking branch 'upstream/master'
Niharikadutta Oct 8, 2020
cb6aa7a
Merge remote-tracking branch 'upstream/master'
Niharikadutta Oct 12, 2020
cbe6e50
Merge branch 'master' of github.com:Niharikadutta/spark
Niharikadutta Oct 12, 2020
3a04b19
Merge remote-tracking branch 'upstream/master'
Niharikadutta Oct 12, 2020
9377692
removing duplicate docs
Niharikadutta Oct 12, 2020
1295934
reverting table changes
Niharikadutta Oct 12, 2020
7497bd7
changes
Niharikadutta Oct 12, 2020
2c498dc
Merge remote-tracking branch 'upstream/master'
Niharikadutta Oct 13, 2020
d19cfb6
Merge remote-tracking branch 'upstream/master'
Niharikadutta Oct 16, 2020
d34188e
Merge branch 'master' of github.com:Niharikadutta/spark
Niharikadutta Oct 16, 2020
5457ffb
Merge remote-tracking branch 'upstream/master'
Niharikadutta Oct 26, 2020
2a44453
Merge remote-tracking branch 'upstream/master'
Niharikadutta Nov 4, 2020
3ec9756
Merge remote-tracking branch 'upstream/master'
Niharikadutta Nov 12, 2020
144233b
Merge remote-tracking branch 'upstream/master'
Niharikadutta Nov 18, 2020
ff38239
resolving merge conflicts
Niharikadutta Nov 18, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

.NET for Apache Spark is compliant with .NET Standard - a formal specification of .NET APIs that are common across .NET implementations. This means you can use .NET for Apache Spark anywhere you write .NET code allowing you to reuse all the knowledge, skills, code, and libraries you already have as a .NET developer.

.NET for Apache Spark runs on Windows, Linux, and macOS using .NET Core, or Windows using .NET Framework. It also runs on all major cloud providers including [Azure HDInsight Spark](deployment/README.md#azure-hdinsight-spark), [Amazon EMR Spark](deployment/README.md#amazon-emr-spark), [AWS](deployment/README.md#databricks) & [Azure](deployment/README.md#databricks) Databricks.
.NET for Apache Spark runs on Windows, Linux, and macOS using .NET Core, or Windows using .NET Framework. It also runs on all major cloud providers including [Azure HDInsight Spark](https://docs.microsoft.com/en-us/dotnet/spark/tutorials/hdinsight-deployment), [Amazon EMR Spark](https://docs.microsoft.com/en-us/dotnet/spark/tutorials/amazon-emr-spark-deployment), [AWS](deployment/README.md#databricks), [Azure Databricks](https://docs.microsoft.com/en-us/dotnet/spark/tutorials/databricks-deployment) & [Azure Synapse Analytics](https://docs.microsoft.com/en-us/azure/synapse-analytics/spark/spark-dotnet).

**Note**: We currently have a Spark Project Improvement Proposal JIRA at [SPIP: .NET bindings for Apache Spark](https://issues.apache.org/jira/browse/SPARK-27006) to work with the community towards getting .NET support by default into Apache Spark. We highly encourage you to participate in the discussion.

Expand Down Expand Up @@ -58,9 +58,9 @@

## Get Started
These instructions will show you how to run a .NET for Apache Spark app using .NET Core.
- [Windows Instructions](docs/getting-started/windows-instructions.md)
- [Ubuntu Instructions](docs/getting-started/ubuntu-instructions.md)
- [MacOs Instructions](docs/getting-started/macos-instructions.md)
- [Windows Instructions](https://docs.microsoft.com/en-us/dotnet/spark/tutorials/get-started?tabs=windows)
- [Ubuntu Instructions](https://docs.microsoft.com/en-us/dotnet/spark/tutorials/get-started?tabs=linux)
- [MacOs Instructions](https://docs.microsoft.com/en-us/dotnet/spark/tutorials/get-started?tabs=linux)

## Build Status

Expand All @@ -75,8 +75,8 @@ Building from source is very easy and the whole process (from cloning to being a

| | | Instructions |
| :---: | :--- | :--- |
| ![Windows icon](docs/img/windows-icon-32.png) | **Windows** | <ul><li>Local - [.NET Framework 4.6.1](docs/building/windows-instructions.md#using-visual-studio-for-net-framework-461)</li><li>Local - [.NET Core 3.1](docs/building/windows-instructions.md#using-net-core-cli-for-net-core)</li><ul> |
| ![Ubuntu icon](docs/img/ubuntu-icon-32.png) | **Ubuntu** | <ul><li>Local - [.NET Core 3.1](docs/building/ubuntu-instructions.md)</li><li>[Azure HDInsight Spark - .NET Core 3.1](deployment/README.md)</li></ul> |
| ![Windows icon](docs/img/windows-icon-32.png) | **Windows** | <ul><li>Local - [.NET Framework 4.6.1](https://docs.microsoft.com/en-us/dotnet/spark/how-to-guides/windows-instructions#using-visual-studio-for-net-framework)</li><li>Local - [.NET Core 3.1](https://docs.microsoft.com/en-us/dotnet/spark/how-to-guides/windows-instructions#using-net-core-cli-for-net-core)</li><ul> |
| ![Ubuntu icon](docs/img/ubuntu-icon-32.png) | **Ubuntu** | <ul><li>Local - [.NET Core 3.1](https://docs.microsoft.com/en-us/dotnet/spark/how-to-guides/ubuntu-instructions)</li><li>[Azure HDInsight Spark - .NET Core 3.1](https://docs.microsoft.com/en-us/dotnet/spark/tutorials/hdinsight-deployment)</li></ul> |

<a name="samples"></a>
## Samples
Expand Down Expand Up @@ -144,6 +144,10 @@ We welcome contributions to both categories!
</tr>
</table>

## Learn More

To learn more about some features of .NET for Apache Spark, please visit [the official .NET documentation](https://docs.microsoft.com/en-us/dotnet/spark/).

## Contributing

We welcome contributions! Please review our [contribution guide](CONTRIBUTING.md).
Expand Down
92 changes: 0 additions & 92 deletions docs/broadcast-guide.md

This file was deleted.

218 changes: 0 additions & 218 deletions docs/building/ubuntu-instructions.md

This file was deleted.

Loading