Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 0.12.6 #132

Merged
merged 10 commits into from
Feb 5, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 5 additions & 9 deletions .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
# This workflows will upload a Python Package using Twine when a release is created
# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries

name: Upload Python Package

on:
Expand All @@ -9,9 +6,10 @@ on:

jobs:
deploy:

name: upload release to PyPI
runs-on: ubuntu-latest

permissions:
id-token: write
steps:
- uses: actions/checkout@v2
- name: Set up Python
Expand All @@ -23,9 +21,7 @@ jobs:
python -m pip install --upgrade pip
pip install setuptools wheel twine
- name: Build and publish
env:
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
python setup.py sdist bdist_wheel
twine upload dist/*
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
38 changes: 0 additions & 38 deletions .github/workflows/run-codecov.yml

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/run-pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
python-version: ["3.8", "3.11"]
python-version: ["3.8", "3.12"]
os: [ubuntu-latest]

steps:
Expand Down
4 changes: 4 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,10 @@ geofetch -i GSE95654 --just-metadata
geofetch -i GSE95654 --processed --just-metadata
```


⁣**Note:** We ensure that GEOfetch is compatible with Unix, Linux, and Mac OS X.
However, due to dependencies, some features of GEOfetch may not be available on Windows.
khoroshevskyi marked this conversation as resolved.
Show resolved Hide resolved

### Check out what exactly argument you want to use to download data:

![](./img/arguments_outputs.svg)
Expand Down
3 changes: 3 additions & 0 deletions docs/changelog.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# Changelog

## [0.12.6] -- 2024-01-18
- Updated support for Windows (Some of the functionality could not be available on Windows)

## [0.12.5] -- 2023-11-29
- Fixed bug, where description was not populated in PEP

Expand Down
12 changes: 9 additions & 3 deletions docs_jupyter/python-usage.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,13 @@
}
],
"source": [
"geof = Geofetcher(processed=True, data_source=\"all\", const_limit_project = 20, const_limit_discard = 500, attr_limit_truncate = 10000 )"
"geof = Geofetcher(\n",
" processed=True,\n",
" data_source=\"all\",\n",
" const_limit_project=20,\n",
" const_limit_discard=500,\n",
" attr_limit_truncate=10000,\n",
")"
]
},
{
Expand Down Expand Up @@ -418,7 +424,7 @@
}
],
"source": [
"len(projects['GSE95654_samples'].samples)"
"len(projects[\"GSE95654_samples\"].samples)"
]
},
{
Expand Down Expand Up @@ -684,7 +690,7 @@
}
],
"source": [
"projects['GSE95654_samples'].sample_table.iloc[:15 , :5]"
"projects[\"GSE95654_samples\"].sample_table.iloc[:15, :5]"
]
}
],
Expand Down
2 changes: 1 addition & 1 deletion geofetch/_version.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "0.12.5"
__version__ = "0.12.6"
47 changes: 31 additions & 16 deletions geofetch/geofetch.py
Original file line number Diff line number Diff line change
Expand Up @@ -372,10 +372,17 @@ def fetch_all(self, input: str, name: str = None) -> Union[NoReturn, peppy.Proje
# check to make sure prefetch is callable
if not self.just_metadata and not self.processed:
if not is_command_callable("prefetch"):
raise SystemExit(
"To download raw data You must first install the sratoolkit, with prefetch in your PATH."
" Installation instruction: http://geofetch.databio.org/en/latest/install/"
)
if os.name == "nt":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if its windows, we don't prefetch, and just warn them instead of exiting?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, because I don't know how to check if prefetch exist in windows

Copy link
Member

@nleroy917 nleroy917 Jan 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use where? Its available on windows XP and above it seems: https://stackoverflow.com/questions/304319/is-there-an-equivalent-of-which-on-the-windows-command-line

_LOGGER.warning(
"GEOfetch is not checking if prefetch is installed on Windows,"
" please make sure it is installed and in your PATH, otherwise "
"it will not be possible to download raw data."
)
else:
raise SystemExit(
"To download raw data You must first install the sratoolkit, with prefetch in your PATH."
" Installation instruction: http://geofetch.databio.org/en/latest/install/"
)

acc_GSE_list = parse_accessions(
input, self.metadata_expanded, self.just_metadata
Expand Down Expand Up @@ -1036,7 +1043,7 @@ def _write_processed_annotation(
)

if not just_object:
with open(file_annotation_path, "w") as m_file:
with open(file_annotation_path, "w", encoding="utf-8") as m_file:
dict_writer = csv.DictWriter(m_file, processed_metadata[0].keys())
dict_writer.writeheader()
dict_writer.writerows(processed_metadata)
Expand Down Expand Up @@ -1789,15 +1796,22 @@ def _download_processed_file(self, file_url: str, data_folder: str) -> bool:
return True

except IOError as e:
_LOGGER.error(str(e))
# The server times out if we are hitting it too frequently,
# so we should sleep a bit to reduce frequency
sleeptime = (ntry + 1) ** 3
_LOGGER.info(f"Sleeping for {sleeptime} seconds")
time.sleep(sleeptime)
ntry += 1
if ntry > 4:
raise e
if os.name == "nt":
_LOGGER.error(f"{e}")
raise OSError(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we raise an error here? I feel like we can just check for the wget in the beginning and handle it from there, no?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, here..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where wget? Maybe

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you try to execute wget --version at the start and mark it unavailable if it raises an error?
In Powershell wget is aliased to Invoke-WebRequest so if you type wget, something will happen but it's not really wget (different command line args etc.) I don't know if Python will cause Powershell commands to be executed or not, but if it does, that could also cause confusion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know if any of this is a problem when using WSL? I thought WSL leveled the playing field when it came to posix compatibility with windows. Unfortunately I don't have immediate access to a Windows machine to test some of these things.

But I kind of agree with @pedro-w in that it should be pretty easy to do an initial check at the beginning to see if wget is available at all and then raise errors/move forward from there

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does appear to work for me under WSL but I only ran one command and saw that it started downloading files (comment)
... but WSL could be one of a number of distros and might not have wget installed (same as Linux, actually)

"Windows may not have wget command. "
"Check if `wget` command is installed correctly."
)
else:
_LOGGER.error(str(e))
# The server times out if we are hitting it too frequently,
# so we should sleep a bit to reduce frequency
sleeptime = (ntry + 1) ** 3
_LOGGER.info(f"Sleeping for {sleeptime} seconds")
time.sleep(sleeptime)
ntry += 1
if ntry > 4:
raise e

def _get_SRA_meta(self, file_gse_content: list, gsm_metadata, file_sra=None):
"""
Expand Down Expand Up @@ -1865,12 +1879,13 @@ def _get_SRA_meta(self, file_gse_content: list, gsm_metadata, file_sra=None):
else:
# open existing annotation
_LOGGER.info("Found SRA metadata, opening..")
with open(file_sra, "r") as m_file:
with open(file_sra, "r", encoding="UTF-8") as m_file:
reader = csv.reader(m_file)
file_list = []
srp_list = []
for k in reader:
file_list.append(k)
if k:
file_list.append(k)
for value_list in file_list[1:]:
srp_list.append(dict(zip(file_list[0], value_list)))

Expand Down
2 changes: 1 addition & 1 deletion geofetch/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ def fetch_metadata(
os.makedirs(dirpath)

# save file:
with open(outpath, "w") as f:
with open(outpath, "w", encoding="utf-8") as f:
f.write(result_text)

return result_list
Expand Down
3 changes: 1 addition & 2 deletions requirements/requirements-all.txt
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
attmap>=0.1.8
colorama>=0.3.9
logmuse>=0.2.6
ubiquerg>=0.6.2
requests>=2.28.1
xmltodict>=0.13.0
pandas>=1.5.3
peppy>=0.35.3
peppy>=0.40.0
rich>=12.5.1
coloredlogs>=15.0.1
Loading