Selefra Code Reading

In the previous article, we discussed how Selefra utilizes Provider to fetch data. Now, let's delve into the details of how Selefra reads YAML code.

Code Block Reading Methods

As a Policy-as-Code tool, Selefra plays a crucial role in code reading. Simple YAML parsing is not sufficient due to the interdependencies and associations between code blocks. We need to read different sections of code separately, establish connections based on keywords, and merge them accordingly. Additionally, it's essential to perform validity checks on the code and report any errors in location and content. Therefore, we need to handle this functionality with special treatment.

Let's first understand the basic methods for reading code blocks and performing validity checks during the reading process.

Selefra and Provider Modules

Selefra and Provider modules are relatively straightforward, with no complex compliance relationships. We only need to define the structure and perform checks. Here's an example code snippet showcasing the Provider module:

// Insert code model here
type SelefraBlock struct {

    // Name of project
    Name string `yaml:"name,omitempty" mapstructure:"name,omitempty"`

    // selefra CloudBlock-related configuration
    CloudBlock *CloudBlock `yaml:"cloud,omitempty" mapstructure:"cloud,omitempty"`

    OpenaiApiKey string `yaml:"openai_api_key,omitempty" mapstructure:"openai_api_key,omitempty"`
    OpenaiMode   string `yaml:"openai_mode,omitempty" mapstructure:"openai_mode,omitempty"`
    OpenaiLimit  uint64 `yaml:"openai_limit,omitempty" mapstructure:"openai_limit,omitempty"`

    // The version of the cli used by the project
    CliVersion string `yaml:"cli_version,omitempty" mapstructure:"cli_version,omitempty"`

    // Global log level. This level is used when the provider does not specify a log level
    LogLevel string `yaml:"log_level,omitempty" mapstructure:"log_level,omitempty"`

    //What are the providers required for operation
    RequireProvidersBlock RequireProvidersBlock `yaml:"providers,omitempty" mapstructure:"providers,omitempty"`

    // The configuration required to connect to the database
    ConnectionBlock *ConnectionBlock `yaml:"connection,omitempty" mapstructure:"connection,omitempty"`

    *LocatableImpl `yaml:"-"`
}

We have defined the structure, and for each code block, we have defined the parseSelefraBlock method within the YamlFileToModuleParser structure to load and validate the Selefra code blocks.

func (x *YamlFileToModuleParser) parseSelefraBlock(selefraBlockKeyNode, selefraBlockValueNode *yaml.Node, diagnostics *schema.Diagnostics) *module.SelefraBlock {

    blockPath := SelefraBlockFieldName

    // Type check
    if selefraBlockValueNode.Kind != yaml.MappingNode {
        diagnostics.AddDiagnostics(x.buildNodeErrorMsgForMappingType(selefraBlockValueNode, blockPath))
        return nil
    }

    toMap, d := x.toMap(selefraBlockValueNode, blockPath)
    diagnostics.AddDiagnostics(d)
    if utils.HasError(d) {
        return nil
    }

    selefraBlock := module.NewSelefraBlock()
    for key, entry := range toMap {
        switch key {

        case SelefraBlockNameFieldName:
            selefraBlock.Name = x.parseStringValueWithDiagnosticsAndSetLocation(selefraBlock, SelefraBlockNameFieldName, entry, blockPath, diagnostics)

        // Omit some code
        default:
            diagnostics.AddDiagnostics(x.buildNodeErrorMsgForUnSupport(entry.key, entry.value, fmt.Sprintf("%s.%s", blockPath, key)))
        }
    }

    if selefraBlock.IsEmpty() {
        return nil

    }

    // Set code location
    x.setLocationKVWithDiagnostics(selefraBlock, "", blockPath, newNodeEntry(selefraBlockKeyNode, selefraBlockValueNode), diagnostics)

    return selefraBlock
}

During the parsing of each data, we use predefined validation methods. Taking the parseStringValueWithDiagnosticsAndSetLocation method as an example:

func (x *YamlFileToModuleParser) parseStringValueWithDiagnosticsAndSetLocation(block module.Block, fieldName string, entry *nodeEntry, blockBasePath string, diagnostics *schema.Diagnostics) string {
    valueString := x.parseStringWithDiagnostics(entry.value, blockBasePath+"."+fieldName, diagnostics)

    if entry.key != nil {
        x.setLocationWithDiagnostics(block, fieldName+module.NodeLocationSelfKey, blockBasePath, entry.key, diagnostics)
    }

    x.setLocationWithDiagnostics(block, fieldName+module.NodeLocationSelfValue, blockBasePath, entry.value, diagnostics)

    return valueString
}

func (x *YamlFileToModuleParser) setLocationWithDiagnostics(block module.Block, relativeYamlSelectorPath, fullYamlSelectorPath string, node *yaml.Node, diagnostics *schema.Diagnostics) {
    location := module.BuildLocationFromYamlNode(x.yamlFilePath, fullYamlSelectorPath, node)
    err := block.SetNodeLocation(relativeYamlSelectorPath, location)
    if err != nil {
        diagnostics.AddErrorMsg("YamlFileToModuleParser error, build location for file %s %s error: %s", x.yamlFilePath, fullYamlSelectorPath, err.Error())
    }
}

In this method, we validate the code value's validity and report any errors and modification suggestions to the user when it's invalid.

Complex Code Block: Module Block

In the Module block, we require additional method invocations for loading. We load sub-modules based on the uses field within the code block and establish associations using names and parent-child module pointers for subsequent code execution.

// Load sub-modules
subModuleSlice, loadSuccess := x.loadSubModules(ctx, finalModule.ModulesBlock)
if !loadSuccess {
    return nil, false
}
finalModule.SubModules = subModuleSlice
finalModule.Source = x.options.Source
finalModule.ModuleLocalDirectory = x.options.ModuleDirectory
finalModule.DependenciesPath = x.options.DependenciesTree

These are the key operations involved in reading Selefra code. In the next article, we will explore how Selefra executes rules, queries non-compliant entries, and delivers the expected results to the user.

Thank you for reading!

Please follow our project and provide your star:

github.com/selefra/selefra