When do I get an advantage from git and bash?#
so far we have used git and bash to accomplish familiar goals, and git and bash feel like just extra work for familiar goals.
Today, we will start to see why git and bash are essential skills: they give you efficiency gains and time traveling super powers (within your work, only, sorry)
Setting the stage#
To prepare for today’s class you examined an open source project.
We noticed that while the contents inside and the distribution of languages used as well as the specific code, or the content of the files was all different, a lot of the organization was similar.
Most had certain community health files and basic info files:
CONTRIBUTINGCODE_OF_CONDUCTREADME.MDLICENSEGOVERNANCE.MD
Important references#
Use these for checking facts and resources.
Setup#
First, we’ll go back to our github inclass folder, locally
cd gh-inclass-brownsarahm/
And confirm we are where we want to be
pwd
/Users/brownsarahm/Documents/inclass/systems/gh-inclass-brownsarahm
Next get the files for today’s activity:
Find your PR that I opened for you today that has the title, “9/19 in class activity”
Mark it ready for review to change from draft
Merge it
Note
in the issue PR it has the info about draft PRs in a link
Now we pull to get the files locally:
git pull
remote: Enumerating objects: 20, done.
remote: Counting objects: 100% (20/20), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 19 (delta 1), reused 17 (delta 0), pack-reused 0 (from 0)
Unpacking objects: 100% (19/19), 2.47 KiB | 126.00 KiB/s, done.
From https://github.com/compsys-progtools/gh-inclass-brownsarahm
0c12714..991ee65 main -> origin/main
* [new branch] organizing_ac -> origin/organizing_ac
Already up to date.
Note how many things it got
and check out status
git status
On branch fun_fact
Your branch is up to date with 'origin/fun_fact'.
nothing to commit, working tree clean
we are still on fun_fact
How does git store information?#
In class, I received the following question
Why are we on the
fun_factbranch, notmain?
This is because of how a branch is implemented and how git runs.
Instead of storing important things in variables that only have scope of how long the program is active git stores its imporant information in files that it reads each time we run a command.
All of git’s data is in the .git directory.
ls .git
COMMIT_EDITMSG ORIG_HEAD description info packed-refs
FETCH_HEAD REBASE_HEAD hooks logs refs
HEAD config index objects
We will learn more about these files later, but looking at one helps us answer the question
First we will look at the HEAD file
cat .git/HEAD
ref: refs/heads/fun_fact
HEAD is a pointer to the currently checked out branch.
The other files with HEAD in their name are similarly pointers to other references, named corresponding to other things.
Next we switch to main, since we want to be there for the files we merged anyway
git checkout main
Switched to branch 'main'
Your branch is behind 'origin/main' by 2 commits, and can be fast-forwarded.
(use "git pull" to update your local branch)
Now we can look at the HEAD file again
cat .git/HEAD
ref: refs/heads/main
it changed! because one the the things that git checkout does is update the head pointer.
So HEAD is a pointer to the branch, but the branch is also a pointer to a commit
cat .git/refs/heads/main
0c1271483e62e69b8b3fc329203617b7093413df
that file has only the hash of a commit
Each of us will have a unique hash, we’ll learn more about why for next week
but if we use git log
git log
Important
git log starts a program that you can exit with the q key.
The branch pointer matches the lastcommit where the HEAD and main pointers are. (git log also reads those files…)
commit 0c1271483e62e69b8b3fc329203617b7093413df (HEAD -> main, my_branch_checkedoutb, my_branch)
Merge: 0e7c990 c7375fa
Author: Sarah Brown <brownsarahm@uri.edu>
Date: Tue Sep 17 12:50:51 2024 -0400
Note
This is truncated output for brevity
Now again, we can use git status
git status
On branch main
Your branch is behind 'origin/main' by 2 commits, and can be fast-forwarded.
(use "git pull" to update your local branch)
nothing to commit, working tree clean
We’re behind, so the files we merged into main are not here yet
ls
README.md about.md
so, we will pull
git pull
Updating 0c12714..991ee65
Fast-forward
API.md | 1 +
CONTRIBUTING.md | 1 +
LICENSE.md | 1 +
_config.yml | 1 +
_toc.yml | 1 +
abstract_base_class.py | 1 +
alternative_classes.py | 1 +
example.md | 1 +
helper_functions.py | 1 +
important_classes.py | 1 +
philosophy.md | 1 +
scratch.ipynb | 1 +
setup.py | 1 +
tests_alt.py | 1 +
tests_helpers.py | 1 +
tests_imp.py | 1 +
tsets_abc.py | 1 +
17 files changed, 17 insertions(+)
create mode 100644 API.md
create mode 100644 CONTRIBUTING.md
create mode 100644 LICENSE.md
create mode 100644 _config.yml
create mode 100644 _toc.yml
create mode 100644 abstract_base_class.py
create mode 100644 alternative_classes.py
create mode 100644 example.md
create mode 100644 helper_functions.py
create mode 100644 important_classes.py
create mode 100644 philosophy.md
create mode 100644 scratch.ipynb
create mode 100644 setup.py
create mode 100644 tests_alt.py
create mode 100644 tests_helpers.py
create mode 100644 tests_imp.py
create mode 100644 tsets_abc.py
Organizing a project (working with files)#
A common question is about how to organize projects. While our main focus
in this class session is the bash commands to do it, the task that we are
going to do is to organize a hypothetical python project
Put another way, we are using organizing a project as the context to motivate practicing with bash commands for moving files.
A different way to learn this might be to through a slide deck that lists commands and describes what each one does and then have examples at the end. Instead, we are going to focus on organizing files, and I will introduce the commands we need along the ways.
next we are going to pretend we worked on the project and made a bunch of files
I gave a bunch of files, each with a short phrase in them.
none of these are functional files
the phrases mean you can inspect them on the terminal
Note
file extensions are for people; they do not specify what the file is actually written like
these are all actually plain text files
For example
cat API.md
jupyterbook file to generate api documentation
But our older files are as expected
cat README.md
# GitHub Practice
Name: Sarah Brown
Note
Here we talked about how those small files do not have a new line character at the end
Note
On Windows, students ge the warning about CRLF and LF
There is setting in git that controls it. For a good explanation, consider this stack overflow question.-,How%20autocrlf%20works%3A,-core.autocrlf%3Dtrue) about how it works.
GitHub also has a whole file on how to work with this.
Wikipedia’s history in the Newline article notes that the CRLF that Windows uses actually comes from the era of teletype machines. Unix adopted LF alone and Apple (pre OsX) used CR alone.
How does cat work?#
cat --help
cat: illegal option -- -
usage: cat [-belnstuv] [file ...]
it doesn’t have a help option, but the error still gives us the beginning
of its documentation. it says there are options we can provide [-belnstuv]
and then we pass it a file [file...]
If we run it without an explicit file, it uses STDIN which we can see by trying it, or from its docs
cat
each time we type things into STDIN, it outputs that
lksdfldkfds
lksdfldkfds
^C
we can ust CTRL + C (^C means that is what I pressed)
to exit.
Files, Redirects, git restore#
we will work on a branch so that we can easily recover from any mistakes
git checkout -b organization
Switched to a new branch 'organization'
and check status to ensure we are where we want to be and note that no files have been changed
git status
On branch organization
nothing to commit, working tree clean
Let’s review what is in the README
cat README.md
# GitHub Practice
Name: Sarah Brown
Echo repeats things we pass into it
echo "its finally fall"
its finally fall
since the echo program writes to the STDOUT file, we can change it to write to another
file by redirecting it to another file.
echo "its finally fall" > README.md
There is no outut of this command
but we can look at the file
cat README.md
its finally fall
It wrote over. This would be bad, we lost content, but this is what git is for!
It is very very easy to undo work since our last commit.
This is good for times when you have something you have an idea and you do not know if it is going to work, so you make a commit before you try it. Then you can try it out. If it doesn’t work you can undo and go back to the place where you made the commit.
As always we start by checking in
git status
On branch organization
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: README.md
no changes added to commit (use "git add" and/or "git commit -a")
it gives us a reminder, use "git restore <file>..." to discard changes in working directory
So we do that:
git restore README.md
then check in with git again
git status
On branch organization
nothing to commit, working tree clean
and check the file
cat README.md
# GitHub Practice
Name: Sarah Brown
back as we wanted!
Typically, when we write to a file, in programming, we also have to tell it what mode to open the file with, and some options are:
read
write
append
This could be familiar from:
fopenin Cor
openin Python
References
C is not an open source language in the typical sense so there is no “official” C docs
We can also redirect the contents of a command from stdout to a file in bash. Like file operations while programming there is a similar concept to this mode.
There are two types of redirects, like there are two ways to write to a file, more generally:
overwrite (
>)append (
>>)
We can add contents to files with echo and >>
echo "its finally fall" >> README.md
Then we check the contents of the file and we see that the new content is there.
cat README.md
# GitHub Practice
Name: Sarah Brown
its finally fall
We can redirect other commands too:
git status > curgit
we see this created a new file
ls
API.md abstract_base_class.py scratch.ipynb
CONTRIBUTING.md alternative_classes.py setup.py
LICENSE.md curgit tests_alt.py
README.md example.md tests_helpers.py
_config.yml helper_functions.py tests_imp.py
_toc.yml important_classes.py tsets_abc.py
about.md philosophy.md
and we can look at its contents too
cat curgit
On branch organization
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: README.md
Untracked files:
(use "git add <file>..." to include in what will be committed)
curgit
no changes added to commit (use "git add" and/or "git commit -a")
this is not a file we actually want, which gives us a chance to learn another new bash command:
rm for remove
rm curgit
Note that this is a true, full, and complete DELETE, this does not put the file in your recycling bin or the apple trash can that you can recover the file from, it is gone for real.
We will see soon a way around this, because git can help.
use rm with great care
git status
On branch organization
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: README.md
no changes added to commit (use "git add" and/or "git commit -a")
Now we have made some changes we want, so let’s commit our changes.
git commit -a -m "add a note"
[organization 72b85c7] add a note
1 file changed, 1 insertion(+)
git status
On branch organization
nothing to commit, working tree clean
git status
Now we will add some text to the readme
echo "|file | contents |
> > | --| -- |
> > | abstract_base_class.py | core abstract classes for the project |
> > | helper_functions.py | utitly funtions that are called by many classes |
> > | important_classes.py | classes that inherit from the abc |
> > | alternative_classes.py | classes that inherit from the abc |
> > | LICENSE.md | the info on how the code can be reused|
> > | CONTRIBUTING.md | instructions for how people can contribute to the project|
> > | setup.py | file with function with instructions for pip |
> > | test_abc.py | tests for constructors and methods in abstract_base_class.py|
> > | tests_helpers.py | tests for constructors and methods in helper_functions.py|
> > | tests_imp.py | tests for constructors and methods in important_classes.py|
> > | tests_alt.py | tests for constructors and methods in alternative_classes.py|
> > | API.md | jupyterbook file to generate api documentation |
> > | _config.yml | jupyterbook config for documentation |
> > | _toc.yml | jupyter book toc file for documentation |
> > | philosophy.md | overview of how the code is organized for docs |
> > | example.md | myst notebook example of using the code |
> > | scratch.ipynb | jupyter notebook from dev |" >> README.md
cat README.md
# GitHub Practice
Name: Sarah Brown
its finally fall
|file | contents |
> | --| -- |
> | abstract_base_class.py | core abstract classes for the project |
> | helper_functions.py | utitly funtions that are called by many classes |
> | important_classes.py | classes that inherit from the abc |
> | alternative_classes.py | classes that inherit from the abc |
> | LICENSE.md | the info on how the code can be reused|
> | CONTRIBUTING.md | instructions for how people can contribute to the project|
> | setup.py | file with function with instructions for pip |
> | test_abc.py | tests for constructors and methods in abstract_base_class.py|
> | tests_helpers.py | tests for constructors and methods in helper_functions.py|
> | tests_imp.py | tests for constructors and methods in important_classes.py|
> | tests_alt.py | tests for constructors and methods in alternative_classes.py|
> | API.md | jupyterbook file to generate api documentation |
> | _config.yml | jupyterbook config for documentation |
> | _toc.yml | jupyter book toc file for documentation |
> | philosophy.md | overview of how the code is organized for docs |
> | example.md | myst notebook example of using the code |
> | scratch.ipynb | jupyter notebook from dev |
this explains each file a little bit more than the name of it does. We see there are sort of 5 groups of files:
about the project/repository
code that defines a python module
test code
documentation
extra files that “we know” we can delete.
We also learn something about bash:
using the open quote " then you stay inside that until you close it. when you press enter the command does not run until after you close the quotes
Finally, we will commit the changes
git commit -a -m 'explain files'
[organization f17e276] explain files
1 file changed, 19 insertions(+)
Prepare for Next Class#
Bring git questions or scenarios you want to be able to solve to class on Thursday (in your mind or comment here if that helps you remember)
Try read and understand the workflow files in your KWL repo, the goal is not to be sure you understand every step, but to get an idea about the big picture ideas and just enough to complete the following. Try to modify files, on a prepare branch, so that your name is already filled in and
VioletVexis already requested as a reviewer when your experience badge (inclass) action runs. We will give the answer in class, but especially do not do this step on the main branch it could break your action. Hints: Look for bash commands that we have seen before andcpcopies a file.
Badges#
Update your KWL chart with the new items and any learned items.
Clone the course website. Append the commands used and the contents of your
fall2024/.git/configto a terminal_review.md (hint: history outputs recent commands and redirects can work with any command, not only echo). Edit the README.md, commit, and try to push the changes. Describe what the error means and which GitHub Collaboration Feature you think would enable you to push? (answer in theterminal_review.md)
Update your KWL chart with any learned items.
Get set up so that you can contribute to the course website repo from your local system. Note: you can pull from the
compsys-progtools/fall2024repo, but you do not not have push permission, so there is more to do than clone. Append the commands used and the contents of yourfll2024/.git/configto a git-remote-practice.md. Then, using a text editor (or IDE), wrap each log with three backticks to make them fenced code blocks and add headings to the sections.learn about options for how git can display commit history. Try out a few different options. Choose two, write them both to a file (from the command line, not copy&paste), gitlog-compare.md. Then, using a text editor (or IDE), wrap each log with three backticks to make them fenced code blocks and then add text to the file describing a use case where that format in particular would be helpful.
Experience Report Evidence#
Save your history with:
history > activity-2024-09-19.md
Questions After Today’s Class#
Would using > and >> locally create a merge conflict if the edited file also has a merge to it on Github?#
any edit to a file could create a merge conflict, so yes one wiht a redirect can
I would like to learn more about adding text into files, for example if you can add text in between specific lines when writing to a file#
This requires more complex commands than we will use in class time, but it can be done with sed. A tutorial on this could be a good explore badge.
If I forgot to restore something for a couple of days and then chose to restore it a while later would I still be able to or is there a time limit to doing so?#
There is not a time limit, but if you did other operations it can require different commands beside restore
Can bash be used to trigger cursor events like mouse clicks?#
I do not think so, but a headless browser allows you to automate browser operations, including selecting things that are most typically done by users with a mouse click, because in the browser clickable things are labeled in other ways.
This is also a good explore option.
What is different between using echo or nano to add to a file?#
Adding to a file with echo and a redirect is nice for adding small changes, but not extnsive changes or changes to the middle of a file. nano is a full text editor. In class today, echo was an easy way to demonstrate redirects, because it is so simple, but redirects are more powerful with more powerful commands.
A question that I have is about git pull, does it run git fetch beforehand to retrieve new content from the online repo Or is git pull just a completely different pocess to bring down content?#
… precisely,
git pullrunsgit fetchwith the given parameters and then depending on configuration options or command line flags, will call eithergit rebaseorgit mergeto reconcile diverging branches.
example with diagrams in last class notes