Splunk Review

One of my favorite tools over the years has been Splunk. It’s an excellent option for gathering and analyzing log data from a variety of sources. Use it for monitoring, troubleshooting, dashboarding and alerting across apps, DBs, sensors, IoT devices and more. I’ve not had a chance to use it in a while so wanted to review. I accomplished that goal by completing this Lynda course. Some of my notes are below.

Here are some key terms to consider when using Splunk:

  • Apps
    • collections of knowledge objects and customer-designed views and dashboards
  • Host
    • it’s a default field
    • contains host name, or IP of the device that generated the event
    • each event has one
    • Splunk indexer generates this at index time
  • Forwarder
    • grabs data from different sources
    • sends that data into Splunk software
    • acts as an agent for log collection and sends to indexer
  • Index
    • repository for data
    • transforms data into searchable events
    • 2 types
      • events
      • metrics
  • Tag
    • Knowledge object that enables searching for specific values
  • Indexer
    • transforms raw data into events & stores them into an index
    • searches data in response to search requests
  • Event
    • set of values associated with a timestamp
    • single entry
    • can have 1 or more lines
  • Source
    • name of a file, directory, data stream etc
    • from where an event originates

Networking Review – Updated

I needed to review some computer networking concepts this week. I’ve learned and forgotten a lot of this stuff more than once so thought I’d write some of it down here for easy reference. I’ll likely add more as I get to it.

Updated 9/10/20: Just adding a few more bullets from my studies.

  1. DNS
    1. Domain Name System
    2. Matches URLs to IPs
    3. Your computer can’t find ‘wordpress.com’
      1. When you try visiting an address, your local DNS cache will be checked first
      2. If the address is not found, then the resolver server cache will be checked with your ISP
      3. If not found with your ISP, then the DNS root server will be checked to match the URL to the IP
  2. DHCP
    1. Domain Host Control Protocol
    2. At home it lives in your router
      1. In an office it likely has a dedicated server
    3. Provides hosts in the home network with IP addresses
    4. Also provides subnet mask and default gateway
      1. default gateway is your router, your ‘gateway to the internet’
  3. NAT
    1. Network Address Translation
    2. Lives in routers
    3. Translates public & private IP addresses
    4. Inside your network your IPs are private
    5. Outside of your network, IPs are public
  4. MAC Address
    1. Media Access Control address
    2. ensures the physical address of the computer
      1. while IP address uniquely identifies the connection
    3. operates on the data link layer
    4. aka physical address
  5. WPA2
    1. WiFi Protected Access 2
    2. Security setting that lives in your router
      1. Just navigate to your router’s Admin page to check that out
  6. Troubleshooting Network Connectivity Basics (from Windows)
    1. Can you see your router?
      1. Windows key>’cmd’
      2. type ‘ipconfig’
        1. ipconfig is used to display your connection to the internet
      3. look for your gateway/router
        1. will likely be ‘192.168.1.1’
      4. type ‘ping 192.168.1.1’ to check that you can reach your router
      5. You should get back a response like like this:
        • Pinging 192.168.1.1 with 32 bytes of data:
        • Reply from 192.168.1.1: bytes=32 time=3ms TTL=64
      6. If you instead get 0.0.0.0 with ipconfig or the ping, check that your router is on and plugged in etc.
      7. traceroute
        1. coming soon!
  7. SCP
    1. Secure Copy Protocol
    2. Based on SSH
    3. Provides file transfers between hosts on a network
    4. It’s faster than SFTP (Secure FTP)
      1. Though SFTP is better at file management
    5. Has a more efficient transfer algorithm
    6. Bonus item as this is not necessarily related to networking 🙂
  8. Hardware
    1. Hub
      1. Physical Layer device
      2. connects multiple devices to a network
      3. connects computers to a LAN
      4. basically a repeater
    2. Switch
      1. Data Link Layer device
      2. ‘creates’ networks
      3. used filtering and forwarding
      4. a virtual LAN can be created here
      5. allows different network devices to communicate
    3. Router
      1. Network Layer device
      2. allows different networks to communicate
      3. gateway to the network
  9. OSI Model
    1. The OSI model (Open Systems Interconnection) describes the functions of a network system. It’s a more generic description of the TCP/IP Model.
      1. Lots more info here as well!
    2. Within the OSI Model there are 7 layers. I’ve bulleted these out below:
      1. Application Layer
        1. highest level
        2. provides interfaces for users
        3. interacts directly with software applications
        4. uses both TCP/IP and OSI models
        5. presents data to user
        6. ensures receiving device is identified, can be accessed and is available to receive data
        7. ensures effective communication b/n application and network
      2. Presentation Layer
        1. hands data to the application
        2. translates between application and network formats
        3. transforms data into a form that application accepts
        4. de/encryption
        5. data compression
        6. graphic handling
      3. Session Layer
        1. it’s the dialogue controller
        2. establishes, manages, syncs & terminates communication b/n end use application processes
        3. token management
      4. Transport Layer
        1. responsible for end-to-end communication over the network
        2. collects message segments from applications and transmits them on to the network layer
        3. adds a port address to the data packet header
        4. divides messages into smaller segments
      5. Network Layer
        1. allows for connection and transfer of data packets b/n different devices and networks
        2. provides routing paths for data
        3. considered backbone of OSI Model
        4. selects and manages best logical path for transfer of data b/n nodes
        5. assigns destination and source IP addresses to each data segment
        6. responds to requests from transport layer
        7. issues requests to data link layer
      6. Data Link Layer
        1. encodes/decodes data bits
        2. organizes bits into patterns
        3. coordinates data volume
        4. detects and retransmits damaged or lost frames
        5. sub-layers are:
          1. MAC – Media Access Control (see above)
          2. LLC – Logical Link Control
      7. Physical Layer
        1. actual physical connection between devices
        2. info is in the form of bits
        3. transmits individual bits from from one node to the next
        4. receives signal, converts to 1s & 0s and sends them to the Data Link layer for reassembly

Software Test Planning Review

Given that I’ve mostly been straight coding lately, well…, and also enjoying as much of the Seattle Summer as possible, I thought it’d be a good time to do review some QA concepts.

To that end, I found this great course on LinkedIn Learning, Software Testing Foundations: Test Planning. The instructor, Mike Fine, reviews some of the best elements of test planning, such as scope, deliverables, defining the product and test plan as well as communicating your results. I’d definitely recommend this to new or veteran testers as an introduction or review respectively.

#100DaysOfCode Round 2 Goals

  • Update: on Round 2 Goals. I made some progress, but didn’t keep good track of these and got distracted by other projects. Gotta keep trying!

I’m feeling excited about round 2 of #100DaysOfCode and wanted to put down some goals. Here they are:

  1. Brush up on GitCOMPLETE
  2. Create Python ProjectsSorta made progress
    1. Use Python GUI modules to create a Software Testing Quiz
      1. Create a #dataviz of the quiz results
    2. Explore data analytics and build data visualizations with Matplotlib and other modules from discovered datasets
      1. Let’s set a goal of 10, that seems reasonable
    1. Create a test automation suite here, against my WordPress site.
  3. Post here 14 timesHa, not even close
    1. Content can include tutorials, study topics, etc.
  4. Learn some Javascript and D3.jsTouched it a couple times. Will circle back in 2021!
  5. Have Fun!

#100DaysOfCode Retrospective

Wow, it’s hard to believe I’ve completed my first #100DaysOfCode. For some background, I’ve worked as a software tester for many years and have had some coding experience, namely with Ruby and Java for test automation. I decided it was time to level-up in a big way and Python was my language of choice given its broad footprint. My goals were to explore multiple areas of Python, create good study habits and just get more code under my fingers. I’ve definitely accomplished all that and more. While some study sessions were very painful, it’s the ones where things ‘clicked’ that made it worth the effort. I’m looking forward to the next 100 days where I’ll apply what I’ve learned and take on new challenges, building projects and creating more content here as well.

Below I’ve outlined some tips and tricks and just general thoughts on my recent experience. Maybe you will find something helpful?

  • Time Management
    • Break tasks into small chunks and log estimates to set a more realistic pace.
      • There were many days where I would set a goal, only to realize around lunch time that I was nowhere close to where I’d hoped to be. Be honest about what you can realistically accomplish within a given time frame.
        • Example: Maybe a coding tutorial is 30-minutes long. You can of course watch the information go by in that time, but to actively code along, take notes and understand the material, it may take three hours!
      • I liked to use the first and last few minutes of a Pomodoro to review previous or upcoming study material.
    • Decrease interruptions both internal and external
    • Tidy your work area. Keep only what you need in front of you.
    • Find good background music.
      • For me it’s mostly white noise and ambient stuff.
        • I can’t listen to anything with vocals, at all.
  • Python Resources
  • Python Study Topics
    • ‘Basics’, foundational stuff
    • Pandas
    • NumPy
    • Pytest and Unittest
    • GUI modules
    • Selenium
  • Physical Environment
    • dual mice -> get ambidextrous!
    • ergo keyboard
    • good chair
    • laptop & larger external monitor
  • Health
    • Using the Pomodoro technique helped me to regularly step away from the computer to:
      • say Hi to my wife ❤
      • hydrate
      • stretch
      • sketch or play guitar
    • Get a 2nd mouse!
      • With a little practice you can become at least minimally proficient on your non-dominant hand.
    • Use keyboard shortcuts to avoid excessive mousing!
    • Get 30 minutes of exercise/day
  • Tools
    • For the Browser
      • Marinara: Pomodoro® Assistant
        • This plugin was crucial to successful time management
        • Great breakdown of the approach here!
      • Video Speed Controller
        • This amazing tool enabled me to get through or to specific tutorial sections
        • I would often use the plugin to quickly review content from days prior, getting my brain into learning mode.
    • Coding
      • Jupyter Notebooks
        • I also started using JupyterLab a bit
      • IDEs
        • Pycharm
        • Visual Studio Code
    • Github
      • Save your work!
      • It’s also just good to know version control
      • My study notes are currently a mess and will be organized during the next 100 days 🙂
  • Accomplishments
  • Bonus Studies
    • Adobe CC products
    • I’m also very interested in visual design so learned a lot here as well.
      • Lightroom – mostly for dog photos
      • Premiere Pro – for a gift video I wanted to make
      • Illustrator – to play with logo design, dataviz and layout
    • I even completed this logo design overview.
  • Next Steps!
    • Continue on to Round 2 of 100 Days
      • Focus areas will be:
        • Test Automation
        • DataViz
          • Scary, but want to learn some Javascript and D3.js
    • Post more frequent updates here!

Well, I hope you got something from this review of my experience. Again, if you’ve not yet participated in #100DaysofCode and have been mulling over the idea, I highly recommend doing so. Thanks for reading and follow my progress on Twitter and here for updates. ~~Jeremy

#100DaysOfCode, Almost Done!

So, I’m 80+ days into #100DaysOfCode! I was hoping to post at the midway point but clearly didn’t get to it, as I was focusing on actually coding! Is it too early to start looking forward to round 2? Below is what I’ve touched so far…

  1. Hours and hours with Python across multiple areas:
    1. ‘Foundational’ stuff
    2. GUIs
    3. Automation with Selenium
    4. Data analysis and dataviz, with Pandas and Matplotlib
    5. Jupyter Notebooks and recently JupyterLab
    6. Markup and magic commands 🙂
  2. I’ve successfully completed these courses and worked on various related projects.
    1. Scripting for Testers‘ – LinkedIn Learning
    2. Python Automation and Testing‘ – LinkedIn Learning
    3. Master Data Analysis with Python‘ – Intro to Pandas – Udemy
    4. Postman Crash Course for Beginners‘ – Learn API Testing – Udemy
  3. Tableau, GIS and Adobe Illustrator tutorials for fun!

2020 Goals

I completed most of my 2019 goals, for example – finishing 2 bike races and passing the the Tableau Desktop Specialist Exam. Now to get ready for 2020.

Here are my 2020 goals:  (Updated to reflect Corona Virus lockdowns)

  • Pass the Tableau Desktop Certified Associate
    • Update: We’ll see if I can get to this.
  • Continue learning Python focusing on:
    • Test Automation
    • Data Visualization and Analysis
  • Ride/cycle 2000 total miles
    • road, trail and trainer miles all count toward this goal
    • these miles will coincide with my goal of entering the MFG Cyclocross race series in the fall
      • Update: Given the current pandemic situation, I doubt I’ll be racing this year.
    • Update: As of July 2020 I have about 800 cycling miles. I better ramp things up!
  • Continue studying:
    • SQL  
    • Statistics
    • GIS

Which Viz Type is Best?

It’s important to leverage the best data visualization approach applicable to the task at hand. Here is one of the better guides I’ve found to aid your efforts. Thanks to the Financial Time Visual Journalism team for creating this. Check out their GitHub Toolbox for more!

https://github.com/ft-interactive/chart-doctor/blob/master/visual-vocabulary/Visual-vocabulary.pdf