When do I get an advantage from git and bash?

5. When do I get an advantage from git and bash?#

so far we have used git and bash to accomplish familiar goals, and git and bash feel like just extra work for familiar goals.

Today, we will start to see why git and bash are essential skills: they give you efficiency gains and time traveling super powers (within your work, only, sorry)

5.1. Setting the stage#

To prepare for today’s class you examined an open source project.

We noticed that while the contents inside and the distribution of languages used as well as the specific code, or the content of the files was all different, a lot of the organization was similar.

Most had certain community health files and basic info files:

CONTRIBUTING
CODE_OF_CONDUCT
README.MD
LICENSE
GOVERNANCE.MD

5.2. Important references#

Use these for checking facts and resources.

bash
git

5.3. Setup#

First, we’ll go back to our github inclass folder, locally

cd gh-inclass-brownsarahm/

And confirm we are where we want to be

pwd

/Users/brownsarahm/Documents/inclass/systems/gh-inclass-brownsarahm

Next get the files for today’s activity:

Find your PR that I opened for you today that has the title, “9/19 in class activity”
Mark it ready for review to change from draft
Merge it

Note

in the issue PR it has the info about draft PRs in a link

Now we pull to get the files locally:

git pull

remote: Enumerating objects: 20, done.
remote: Counting objects: 100% (20/20), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 19 (delta 1), reused 17 (delta 0), pack-reused 0 (from 0)
Unpacking objects: 100% (19/19), 2.47 KiB | 126.00 KiB/s, done.
From https://github.com/compsys-progtools/gh-inclass-brownsarahm
   0c12714..991ee65  main          -> origin/main
 * [new branch]      organizing_ac -> origin/organizing_ac
Already up to date.

Note how many things it got

and check out status

git status

On branch fun_fact
Your branch is up to date with 'origin/fun_fact'.

nothing to commit, working tree clean

we are still on fun_fact

5.4. How does git store information?#

In class, I received the following question

Why are we on the fun_fact branch, not main?

This is because of how a branch is implemented and how git runs.

Instead of storing important things in variables that only have scope of how long the program is active git stores its imporant information in files that it reads each time we run a command. All of git’s data is in the .git directory.

ls .git

COMMIT_EDITMSG	ORIG_HEAD	description	info		packed-refs
FETCH_HEAD	REBASE_HEAD	hooks		logs		refs
HEAD		config		index		objects

We will learn more about these files later, but looking at one helps us answer the question

First we will look at the HEAD file

cat .git/HEAD 

ref: refs/heads/fun_fact

HEAD is a pointer to the currently checked out branch.

The other files with HEAD in their name are similarly pointers to other references, named corresponding to other things.

Next we switch to main, since we want to be there for the files we merged anyway

git checkout main

Switched to branch 'main'
Your branch is behind 'origin/main' by 2 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)

Now we can look at the HEAD file again

cat .git/HEAD 

ref: refs/heads/main

it changed! because one the the things that git checkout does is update the head pointer.

So HEAD is a pointer to the branch, but the branch is also a pointer to a commit

cat .git/refs/heads/main 

0c1271483e62e69b8b3fc329203617b7093413df

that file has only the hash of a commit

Each of us will have a unique hash, we’ll learn more about why for next week

but if we use git log

git log

Important

git log starts a program that you can exit with the q key.

The branch pointer matches the lastcommit where the HEAD and main pointers are. (git log also reads those files…)

commit 0c1271483e62e69b8b3fc329203617b7093413df (HEAD -> main, my_branch_checkedoutb, my_branch)
Merge: 0e7c990 c7375fa
Author: Sarah Brown <brownsarahm@uri.edu>
Date:   Tue Sep 17 12:50:51 2024 -0400

Note

This is truncated output for brevity

Now again, we can use git status

git status

On branch main
Your branch is behind 'origin/main' by 2 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)

nothing to commit, working tree clean

We’re behind, so the files we merged into main are not here yet

ls

README.md	about.md

so, we will pull

git pull

Updating 0c12714..991ee65
Fast-forward
 API.md                 | 1 +
 CONTRIBUTING.md        | 1 +
 LICENSE.md             | 1 +
 _config.yml            | 1 +
 _toc.yml               | 1 +
 abstract_base_class.py | 1 +
 alternative_classes.py | 1 +
 example.md             | 1 +
 helper_functions.py    | 1 +
 important_classes.py   | 1 +
 philosophy.md          | 1 +
 scratch.ipynb          | 1 +
 setup.py               | 1 +
 tests_alt.py           | 1 +
 tests_helpers.py       | 1 +
 tests_imp.py           | 1 +
 tsets_abc.py           | 1 +
 17 files changed, 17 insertions(+)
 create mode 100644 API.md
 create mode 100644 CONTRIBUTING.md
 create mode 100644 LICENSE.md
 create mode 100644 _config.yml
 create mode 100644 _toc.yml
 create mode 100644 abstract_base_class.py
 create mode 100644 alternative_classes.py
 create mode 100644 example.md
 create mode 100644 helper_functions.py
 create mode 100644 important_classes.py
 create mode 100644 philosophy.md
 create mode 100644 scratch.ipynb
 create mode 100644 setup.py
 create mode 100644 tests_alt.py
 create mode 100644 tests_helpers.py
 create mode 100644 tests_imp.py
 create mode 100644 tsets_abc.py

5.5. Organizing a project (working with files)#

A common question is about how to organize projects. While our main focus in this class session is the bash commands to do it, the task that we are going to do is to organize a hypothetical python project

Put another way, we are using organizing a project as the context to motivate practicing with bash commands for moving files.

A different way to learn this might be to through a slide deck that lists commands and describes what each one does and then have examples at the end. Instead, we are going to focus on organizing files, and I will introduce the commands we need along the ways.

next we are going to pretend we worked on the project and made a bunch of files

I gave a bunch of files, each with a short phrase in them.

none of these are functional files
the phrases mean you can inspect them on the terminal

Note

file extensions are for people; they do not specify what the file is actually written like

these are all actually plain text files

For example

cat API.md 

jupyterbook file to generate api documentation

But our older files are as expected

cat README.md 

# GitHub Practice

Name: Sarah Brown

Note

Here we talked about how those small files do not have a new line character at the end

Note

On Windows, students ge the warning about CRLF and LF

There is setting in git that controls it. For a good explanation, consider this stack overflow question.-,How%20autocrlf%20works%3A,-core.autocrlf%3Dtrue) about how it works.

GitHub also has a whole file on how to work with this.

Wikipedia’s history in the Newline article notes that the CRLF that Windows uses actually comes from the era of teletype machines. Unix adopted LF alone and Apple (pre OsX) used CR alone.

more OSs are described in a table

5.5.1. How does `cat` work?#

cat --help

cat: illegal option -- -
usage: cat [-belnstuv] [file ...]

it doesn’t have a help option, but the error still gives us the beginning of its documentation. it says there are options we can provide [-belnstuv] and then we pass it a file [file...]

If we run it without an explicit file, it uses STDIN which we can see by trying it, or from its docs

cat

each time we type things into STDIN, it outputs that

lksdfldkfds
lksdfldkfds
^C

we can ust CTRL + C (^C means that is what I pressed) to exit.

5.6. Files, Redirects, git restore#

we will work on a branch so that we can easily recover from any mistakes

git checkout -b organization

Switched to a new branch 'organization'

and check status to ensure we are where we want to be and note that no files have been changed

git status

On branch organization
nothing to commit, working tree clean

Let’s review what is in the README

cat README.md 

# GitHub Practice

Name: Sarah Brown

Echo repeats things we pass into it

echo "its finally fall"

its finally fall

since the echo program writes to the STDOUT file, we can change it to write to another file by redirecting it to another file.

echo "its finally fall" > README.md 

There is no outut of this command

but we can look at the file

cat README.md 

its finally fall

It wrote over. This would be bad, we lost content, but this is what git is for!

It is very very easy to undo work since our last commit.

This is good for times when you have something you have an idea and you do not know if it is going to work, so you make a commit before you try it. Then you can try it out. If it doesn’t work you can undo and go back to the place where you made the commit.

As always we start by checking in

git status

On branch organization
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   README.md

no changes added to commit (use "git add" and/or "git commit -a")

it gives us a reminder, use "git restore <file>..." to discard changes in working directory

So we do that:

git restore README.md 

then check in with git again

git status

On branch organization
nothing to commit, working tree clean

and check the file

cat README.md 

# GitHub Practice

Name: Sarah Brown

back as we wanted!

Typically, when we write to a file, in programming, we also have to tell it what mode to open the file with, and some options are:

read
write
append

This could be familiar from:

fopen in C
or open in Python

References

C is not an open source language in the typical sense so there is no “official” C docs

We can also redirect the contents of a command from stdout to a file in bash. Like file operations while programming there is a similar concept to this mode.

There are two types of redirects, like there are two ways to write to a file, more generally:

overwrite (>)
append (>>)

We can add contents to files with echo and >>

echo "its finally fall" >> README.md 

Then we check the contents of the file and we see that the new content is there.

cat README.md 

# GitHub Practice

Name: Sarah Brown
its finally fall

We can redirect other commands too:

git status > curgit

we see this created a new file

ls

API.md			abstract_base_class.py	scratch.ipynb
CONTRIBUTING.md		alternative_classes.py	setup.py
LICENSE.md		curgit			tests_alt.py
README.md		example.md		tests_helpers.py
_config.yml		helper_functions.py	tests_imp.py
_toc.yml		important_classes.py	tsets_abc.py
about.md		philosophy.md

and we can look at its contents too

cat curgit

On branch organization
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   README.md

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	curgit

no changes added to commit (use "git add" and/or "git commit -a")

this is not a file we actually want, which gives us a chance to learn another new bash command: rm for remove

rm curgit

Note that this is a true, full, and complete DELETE, this does not put the file in your recycling bin or the apple trash can that you can recover the file from, it is gone for real.

We will see soon a way around this, because git can help.

use rm with great care

git status

On branch organization
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   README.md

no changes added to commit (use "git add" and/or "git commit -a")

Now we have made some changes we want, so let’s commit our changes.

git commit -a -m "add a note"

[organization 72b85c7] add a note
 1 file changed, 1 insertion(+)

git status

On branch organization
nothing to commit, working tree clean

git status

Now we will add some text to the readme

echo "|file | contents |
> > | --| -- |
> > | abstract_base_class.py | core abstract classes for the project |
> > | helper_functions.py | utitly funtions that are called by many classes |
> > | important_classes.py | classes that inherit from the abc |
> > | alternative_classes.py | classes that inherit from the abc |
> > | LICENSE.md | the info on how the code can be reused|
> > | CONTRIBUTING.md | instructions for how people can contribute to the project|
> > | setup.py | file with function with instructions for pip |
> > | test_abc.py | tests for constructors and methods in abstract_base_class.py|
> > | tests_helpers.py | tests for constructors and methods in helper_functions.py|
> > | tests_imp.py | tests for constructors and methods in important_classes.py|
> > | tests_alt.py | tests for constructors and methods in alternative_classes.py|
> > | API.md | jupyterbook file to generate api documentation |
> > | _config.yml | jupyterbook config for documentation |
> > | _toc.yml | jupyter book toc file for documentation |
> > | philosophy.md | overview of how the code is organized for docs |
> > | example.md | myst notebook example of using the code |
> > | scratch.ipynb | jupyter notebook from dev |" >> README.md

cat README.md 

# GitHub Practice

Name: Sarah Brown
its finally fall
|file | contents |
> | --| -- |
> | abstract_base_class.py | core abstract classes for the project |
> | helper_functions.py | utitly funtions that are called by many classes |
> | important_classes.py | classes that inherit from the abc |
> | alternative_classes.py | classes that inherit from the abc |
> | LICENSE.md | the info on how the code can be reused|
> | CONTRIBUTING.md | instructions for how people can contribute to the project|
> | setup.py | file with function with instructions for pip |
> | test_abc.py | tests for constructors and methods in abstract_base_class.py|
> | tests_helpers.py | tests for constructors and methods in helper_functions.py|
> | tests_imp.py | tests for constructors and methods in important_classes.py|
> | tests_alt.py | tests for constructors and methods in alternative_classes.py|
> | API.md | jupyterbook file to generate api documentation |
> | _config.yml | jupyterbook config for documentation |
> | _toc.yml | jupyter book toc file for documentation |
> | philosophy.md | overview of how the code is organized for docs |
> | example.md | myst notebook example of using the code |
> | scratch.ipynb | jupyter notebook from dev |

this explains each file a little bit more than the name of it does. We see there are sort of 5 groups of files:

about the project/repository
code that defines a python module
test code
documentation
extra files that “we know” we can delete.

We also learn something about bash: using the open quote " then you stay inside that until you close it. when you press enter the command does not run until after you close the quotes

Finally, we will commit the changes

git commit -a -m 'explain files'

[organization f17e276] explain files
 1 file changed, 19 insertions(+)

5.7. Summary#

stdout is a file, that is displayed on the terminal
“moving” a file does not re-write the data to a different part of the disk, it updates its adress only
the wildcard operator * allows us to use patterns with bash commands
touch can accept a list of files
bash lists are space delimited without any brackets
the .gitignore file prevents files from being in your repo

5.7.1. New commands#

command	explanation
`echo 'message'`	repeat ‘message’ to stdout
`>`	write redirect
`>>`	append redirect
`rm file`	remove (delete) `file`

5.7.2. New git commands#

command	explanation
`git commit -a -m 'msg'`	the `-a` option adds modified files (but not untracked)

5.8. Prepare for Next Class#

Bring git questions or scenarios you want to be able to solve to class on Thursday (in your mind or comment here if that helps you remember)
Try read and understand the workflow files in your KWL repo, the goal is not to be sure you understand every step, but to get an idea about the big picture ideas and just enough to complete the following. Try to modify files, on a prepare branch, so that your name is already filled in and VioletVex is already requested as a reviewer when your experience badge (inclass) action runs. We will give the answer in class, but especially do not do this step on the main branch it could break your action. Hints: Look for bash commands that we have seen before and cp copies a file.

5.9. Badges#

Review

Update your KWL chart with the new items and any learned items.
Clone the course website. Append the commands used and the contents of your fall2024/.git/configto a terminal_review.md (hint: history outputs recent commands and redirects can work with any command, not only echo). Edit the README.md, commit, and try to push the changes. Describe what the error means and which GitHub Collaboration Feature you think would enable you to push? (answer in the terminal_review.md)

Practice

Update your KWL chart with any learned items.
Get set up so that you can contribute to the course website repo from your local system. Note: you can pull from the compsys-progtools/fall2024 repo, but you do not not have push permission, so there is more to do than clone. Append the commands used and the contents of your fll2024/.git/configto a git-remote-practice.md. Then, using a text editor (or IDE), wrap each log with three backticks to make them fenced code blocks and add headings to the sections.
learn about options for how git can display commit history. Try out a few different options. Choose two, write them both to a file (from the command line, not copy&paste), gitlog-compare.md. Then, using a text editor (or IDE), wrap each log with three backticks to make them fenced code blocks and then add text to the file describing a use case where that format in particular would be helpful.

Hint for working offline

Read about forks and working with remotes.

5.10. Experience Report Evidence#

Save your history with:

history > activity-2024-09-19.md

5.11. Questions After Today’s Class#

5.11.1. Would using > and >> locally create a merge conflict if the edited file also has a merge to it on Github?#

any edit to a file could create a merge conflict, so yes one wiht a redirect can

5.11.2. I would like to learn more about adding text into files, for example if you can add text in between specific lines when writing to a file#

This requires more complex commands than we will use in class time, but it can be done with sed. A tutorial on this could be a good explore badge.

5.11.3. If I forgot to restore something for a couple of days and then chose to restore it a while later would I still be able to or is there a time limit to doing so?#

There is not a time limit, but if you did other operations it can require different commands beside restore

5.11.4. Can bash be used to trigger cursor events like mouse clicks?#

I do not think so, but a headless browser allows you to automate browser operations, including selecting things that are most typically done by users with a mouse click, because in the browser clickable things are labeled in other ways.

This is also a good explore option.

5.11.5. What is different between using echo or nano to add to a file?#

Adding to a file with echo and a redirect is nice for adding small changes, but not extnsive changes or changes to the middle of a file. nano is a full text editor. In class today, echo was an easy way to demonstrate redirects, because it is so simple, but redirects are more powerful with more powerful commands.

5.11.6. A question that I have is about git pull, does it run git fetch beforehand to retrieve new content from the online repo Or is git pull just a completely different pocess to bring down content?#

… precisely, git pull runs git fetch with the given parameters and then depending on configuration options or command line flags, will call either git rebase or git merge to reconcile diverging branches.

—official docs

example with diagrams in last class notes

5.11.7. How do I create a community badge?#

instructions in syllabus

When do I get an advantage from git and bash?

Contents

5. When do I get an advantage from git and bash?#

5.1. Setting the stage#

5.2. Important references#

5.3. Setup#

5.4. How does git store information?#

5.5. Organizing a project (working with files)#

5.5.1. How does cat work?#

5.6. Files, Redirects, git restore#

5.7. Summary#

5.7.1. New commands#

5.7.2. New git commands#

5.8. Prepare for Next Class#

5.9. Badges#

5.10. Experience Report Evidence#

5.11. Questions After Today’s Class#

5.11.1. Would using > and >> locally create a merge conflict if the edited file also has a merge to it on Github?#

5.11.2. I would like to learn more about adding text into files, for example if you can add text in between specific lines when writing to a file#

5.11.3. If I forgot to restore something for a couple of days and then chose to restore it a while later would I still be able to or is there a time limit to doing so?#

5.11.4. Can bash be used to trigger cursor events like mouse clicks?#

5.11.5. What is different between using echo or nano to add to a file?#

5.11.6. A question that I have is about git pull, does it run git fetch beforehand to retrieve new content from the online repo Or is git pull just a completely different pocess to bring down content?#

5.11.7. How do I create a community badge?#

5.5.1. How does `cat` work?#