Structured arrays - use Technical Quotient

2. Understand

What does the dtype([('f0', '<i4'), ('f1', '<i4')]) indicate about the structured array's field byte-order?

Fields are stored in little-endian byte order.

Incorrect; although < does denote little-endian, the correct interpretation is that it specifies native byte order for little-endian systems.

Fields are stored in big-endian byte order.

Incorrect; big-endian byte order is denoted by >, not <.

Fields are stored in native byte order.

Correct; <i4 indicates that fields are stored in little-endian byte order, which is considered native for systems that are little-endian.

The byte order is unspecified.

Incorrect; < specifies the byte order explicitly as little-endian, not unspecified.

3. Apply

You have two structured arrays a and b. Array a has dtype [('a', 'i4'), ('b', 'i4')], and you want to compare it to array b with dtype [('a', 'f4'), ('b', 'i4')]. Write the code snippet to correctly compare these arrays.

import numpy as np\na = np.array([(1, 1), (2, 2)], dtype=[('a', 'i4'), ('b', 'i4')])\nb = np.array([(1.0, 1), (2.5, 2)], dtype=[('a', 'f4'), ('b', 'i4')])\nprint(a == b)

import numpy as np\na = np.array([(1, 1), (2, 2)], dtype=[('a', 'i4'), ('b', 'i4')])\nb = np.array([(1.0, 1), (2.5, 2)], dtype=[('a', 'i4'), ('b', 'i4')])\nprint(np.equal(a, b))

import numpy as np\na = np.array([(1, 1), (2, 2)], dtype=[('a', 'i4'), ('b', 'i4')])\nb = np.array([(1.0, 1), (2.5, 2)], dtype=[('a', 'i4'), ('b', 'i4')])\nprint(a.compare(b))

import numpy as np\na = np.array([(1, 1), (2, 2)], dtype=[('a', 'i4'), ('b', 'i4')])\nb = np.array([(1, 1), (2, 2)], dtype=[('a', 'f4'), ('b', 'i4')])\nprint(a.match(b))

4. Analyze

You're working with two structured arrays representing employee records, where a contains current year data, and b has been updated with next year's projections including salary adjustments. Both arrays use the same field names but different data types for salary to accommodate anticipated changes. When comparing these arrays with the == operator to identify unchanged records, you encounter a TypeError. What is the most likely reason for this error given that structured arrays' dtypes can be promoted for comparison?

Structured arrays require manual iteration for element-wise comparison.
The salary field's data type mismatch prevents comparison.
NumPy arrays cannot be compared using the == operator.

Incorrect; NumPy arrays can indeed be compared using the == operator, including structured arrays under compatible conditions.

The field names between arrays a and b do not match exactly.

6. Remember

What does the join_by function in NumPy's recfunctions module return by default?

list

Incorrect; list is not the type of object returned by NumPy functions which operate on arrays.

ndarray

Incorrect; ndarray is a more generic type and not specific to structured array functions.

recarray

Correct; join_by returns a recarray by default, unless specified otherwise.

masked array

Incorrect; masked array could be an option if usemask=True, but it's not the default return type.

7. Remember

What is the purpose of the structured_to_unstructured function?

Converts a structured array to an unstructured array.

Correct; structured_to_unstructured converts a structured (i.e., with named fields) array to an unstructured array (i.e., plain numeric data).

Sorts a structured array alphabetically by field names.
Merges multiple structured arrays into one.

Incorrect; merging arrays is handled by functions like merge_arrays.

Converts an unstructured array to a structured array.

Incorrect; the reverse operation is performed by unstructured_to_structured.

8. Understand

When using numpy.lib.recfunctions.append_fields, what happens if the dtypes parameter is None?

An error is thrown.

Incorrect; an error is not thrown just because dtypes is None.

The datatypes are estimated from the data.

Correct; if dtypes is None, the datatypes of the new fields are inferred from the provided data.

All fields are set to float64.

Incorrect; not all fields default to float64; they are inferred from the data.

The function fails silently and returns the original array.

9. Understand

What is a benefit of using mask in numpy.lib.recfunctions functions?

Automatically sorts the array by field names.
Reduces the memory usage of the array.
Increases the speed of array operations.
Allows handling of missing or invalid data.

10. Understand

What does the usemask parameter do in functions like join_by?

Converts all numeric fields to masked fields.
Masks the entire array, making it invisible.
Prevents the function from executing.
Returns a masked array to handle missing data.

Correct; usemask=True returns a masked array where missing or invalid data can be handled appropriately.

11. Understand

How does numpy.lib.recfunctions.repack_fields enhance array structure?

It optimizes memory layout by removing field overlaps and adding padding as needed.

Correct; repack_fields improves the memory layout, potentially removing padding and making field offsets contiguous.

It duplicates each field for redundancy.
It encrypts the array data for secure storage.
It converts all fields to use the same datatype.

12. Apply

You need to merge two structured arrays a1 and a2 with fields 'x', 'y', and 'z'. How would you use merge_arrays to accomplish this without using a mask?

np.concatenate((a1, a2))

Incorrect; np.concatenate is not designed for merging structured arrays based on field names.

np.lib.recfunctions.merge_arrays([a1, a2])

Incorrect; while the intent is correct, omitting usemask=False could result in a masked array if missing data exists.

np.merge(a1, a2)

Incorrect; np.merge is not a valid NumPy function for merging arrays.

np.lib.recfunctions.merge_arrays((a1, a2), usemask=False)

Correct; specifies how to merge two structured arrays into one, using merge_arrays with usemask=False.

13. Apply

Given an array:

arr = np.array([(1, 'Alice', 25000), (2, 'Bob', 30000)],\n dtype=[('ID', 'i4'), ('Name', 'U10'), ('Salary', 'f8')])

and a new dataset new_data = [(3, 'Charlie', 28000)] with dtype dtype=[('ID', 'i4'), ('Name', 'U10'), ('Salary', 'f8')]. How do you append new_data to arr using append_fields without creating a mask?

np.lib.recfunctions.append_fields(arr, 'new_data', new_data, usemask=False)

Incorrect; append_fields doesn't append records but fields; thus, this usage is conceptually incorrect.

np.lib.recfunctions.append_fields(arr, ['ID', 'Name', 'Salary'], new_data, usemask=False)

Correct; this option also seems incorrect because append_fields is intended for adding new fields, not records, which makes all options conceptually mismatched for appending records. The appropriate method for adding a new record would involve concatenation or similar operations, not append_fields.

np.concatenate((arr, np.array(new_data, dtype=arr.dtype)))

Incorrect; while np.concatenate is closer to the correct method for appending records, the way structured arrays and their fields are handled requires a different approach, such as directly concatenating the arrays if their dtypes match.

np.append(arr, new_data)

Incorrect; np.append is a method for appending elements but does not specifically handle structured array fields correctly in this context.

14. Apply

You have a structured array employees with fields 'ID', 'Name', and 'Salary'. You need to increase the salary by 10% for all employees. Which approach correctly applies this update?

employees['Salary'] = employees['Salary'] * 1.1

employees = np.lib.recfunctions.append_fields(employees, 'Salary', employees['Salary'] * 1.1, usemask=False)

[for employee in employees: employee['Salary'] *= 1.1]

np.lib.recfunctions.apply_along_fields(lambda x: x['Salary'] * 1.1, employees)

15. Analyze

A data analyst needs to combine employee records from two departments. Each department's data is stored in a structured array with fields 'ID', 'Name', and 'Salary'. The analyst uses join_by('ID', dept1, dept2, jointype='inner') to merge these records. They notice some employee records are missing in the output array. Why are some employee records missing from the merged array?

The function only works with arrays having less than 100 records.
The 'Salary' field data type mismatch caused record loss.
join_by function cannot handle string fields like 'Name'.

Incorrect; join_by can handle string fields, and their presence does not affect record inclusion based on 'ID'.

Only records with matching 'ID' in both arrays are included in the output.

16. Analyze

A team of researchers is working with a structured array containing patient records. Each record includes fields for 'PatientID', 'Age', 'BMI', and 'BloodPressure'. The researchers want to identify outliers in 'BloodPressure' using a threshold value. They decide to use np.lib.recfunctions.apply_along_fields with a custom function is_outlier that checks if 'BloodPressure' exceeds the threshold. Why might some records incorrectly be flagged as outliers?

The custom function is_outlier cannot process structured arrays.
Structured arrays require a different method to apply functions across fields.

Incorrect; apply_along_fields is designed for applying functions across fields in structured arrays, so the method is appropriate but might be misapplied.

Discrepancies in 'BloodPressure' units cause false positives in outlier detection.
The threshold for 'BloodPressure' was set too low, flagging normal values as outliers.

17. Analyze

During data cleaning, a data analyst notices that some entries in a structured array containing employee records have 'Age' fields set to negative values. To correct this, they decide to use np.lib.recfunctions.apply_along_fields to apply a correction function that sets any negative ages to the median age of the dataset. After applying the correction, why do some records still show negative ages?

The function apply_along_fields is not intended for structured arrays, leading to no changes.

Incorrect; apply_along_fields can be used with structured arrays, but the issue lies in how the corrected data is applied.

The correction function returns a new array with updates, not modifying the original array in-place.
Median age calculation is incorrect, resulting in negative values being used as the correction.
Negative ages are a result of incorrect data types that cannot be corrected with apply_along_fields.

1. Remember